[JIRA] [ec2-plugin] (JENKINS-34044) EC2 Plugin: Windows: java.io.IOException: Pipe closed

206 views
Skip to first unread message

matt.mccormick@kitware.com (JIRA)

unread,
Apr 5, 2016, 8:44:03 PM4/5/16
to jenkinsc...@googlegroups.com
Matt McCormick created an issue
 
Jenkins / Bug JENKINS-34044
EC2 Plugin: Windows: java.io.IOException: Pipe closed
Issue Type: Bug Bug
Assignee: Francis Upton
Components: ec2-plugin
Created: 2016/Apr/06 12:43 AM
Environment: Jenkins 1.642.4 (DockerHub 'jenkins' image latest)
Amazon EC2 plugin 1.31
Windows slave Java Runtime 8.0.73
Windows AMI based off MS2012R2-EBS-HVM
Priority: Major Major
Reporter: Matt McCormick

When starting the AWS Windows slave, failure occurs with:

Apr 05, 2016 8:12:55 PM null
FINEST: Node ITKAWSWindows (i-0234e203d06253ae7)(i-0234e203d06253ae7) is still pending/launching, waiting 5s
Apr 05, 2016 8:13:00 PM null
FINEST: Node ITKAWSWindows (i-0234e203d06253ae7)(i-0234e203d06253ae7) is still pending/launching, waiting 5s
Apr 05, 2016 8:13:05 PM null
FINEST: Node ITKAWSWindows (i-0234e203d06253ae7)(i-0234e203d06253ae7) is still pending/launching, waiting 5s
Apr 05, 2016 8:13:10 PM null
FINEST: Node ITKAWSWindows (i-0234e203d06253ae7)(i-0234e203d06253ae7) is still pending/launching, waiting 5s
Apr 05, 2016 8:13:10 PM null
FINER: Node ITKAWSWindows (i-0234e203d06253ae7)(i-0234e203d06253ae7) is ready
ITKAWSWindows (i-0234e203d06253ae7) booted at 1459901636000
Connecting to ec2-54-152-165-170.compute-1.amazonaws.com(54.152.165.170) with WinRM as Administrator
Waiting for WinRM to come up. Sleeping 10s.
Connecting to ec2-54-152-165-170.compute-1.amazonaws.com(54.152.165.170) with WinRM as Administrator
Waiting for WinRM to come up. Sleeping 10s.
Connecting to ec2-54-152-165-170.compute-1.amazonaws.com(54.152.165.170) with WinRM as Administrator
Waiting for WinRM to come up. Sleeping 10s.
Connecting to ec2-54-152-165-170.compute-1.amazonaws.com(54.152.165.170) with WinRM as Administrator
Waiting for WinRM to come up. Sleeping 10s.
Connecting to ec2-54-152-165-170.compute-1.amazonaws.com(54.152.165.170) with WinRM as Administrator
Waiting for WinRM to come up. Sleeping 10s.
Connecting to ec2-54-152-165-170.compute-1.amazonaws.com(54.152.165.170) with WinRM as Administrator
Waiting for WinRM to come up. Sleeping 10s.
Connecting to ec2-54-152-165-170.compute-1.amazonaws.com(54.152.165.170) with WinRM as Administrator
Waiting for WinRM to come up. Sleeping 10s.
Connecting to ec2-54-152-165-170.compute-1.amazonaws.com(54.152.165.170) with WinRM as Administrator
Waiting for WinRM to come up. Sleeping 10s.
Connecting to ec2-54-152-165-170.compute-1.amazonaws.com(54.152.165.170) with WinRM as Administrator
Waiting for WinRM to come up. Sleeping 10s.
Connecting to ec2-54-152-165-170.compute-1.amazonaws.com(54.152.165.170) with WinRM as Administrator
Waiting for WinRM to come up. Sleeping 10s.
Connecting to ec2-54-152-165-170.compute-1.amazonaws.com(54.152.165.170) with WinRM as Administrator
Waiting for WinRM to come up. Sleeping 10s.
Connecting to ec2-54-152-165-170.compute-1.amazonaws.com(54.152.165.170) with WinRM as Administrator
Waiting for WinRM to come up. Sleeping 10s.
Connecting to ec2-54-152-165-170.compute-1.amazonaws.com(54.152.165.170) with WinRM as Administrator
Waiting for WinRM to come up. Sleeping 10s.
Connecting to ec2-54-152-165-170.compute-1.amazonaws.com(54.152.165.170) with WinRM as Administrator
Waiting for WinRM to come up. Sleeping 10s.
Connecting to ec2-54-152-165-170.compute-1.amazonaws.com(54.152.165.170) with WinRM as Administrator
Waiting for WinRM to come up. Sleeping 10s.
Connecting to ec2-54-152-165-170.compute-1.amazonaws.com(54.152.165.170) with WinRM as Administrator
Waiting for WinRM to come up. Sleeping 10s.
Connecting to ec2-54-152-165-170.compute-1.amazonaws.com(54.152.165.170) with WinRM as Administrator
Waiting for WinRM to come up. Sleeping 10s.
Connecting to ec2-54-152-165-170.compute-1.amazonaws.com(54.152.165.170) with WinRM as Administrator
WinRM service responded. Waiting for WinRM service to stabilize on ITKAWSWindows (i-0234e203d06253ae7)
WinRM should now be ok on ITKAWSWindows (i-0234e203d06253ae7)
Connected with WinRM.
Creating tmp directory if it does not exist
slave.jar sent remotely. Bootstrapping it
<===[JENKINS REMOTING CAPACITY]===>Slave.jar version: 2.53.3
This is a Windows slave
ERROR: Connection terminated
java.io.IOException: Pipe closed
	at java.io.PipedInputStream.checkStateForReceive(PipedInputStream.java:260)
	at java.io.PipedInputStream.receive(PipedInputStream.java:226)
	at java.io.PipedOutputStream.write(PipedOutputStream.java:149)
	at java.io.OutputStream.write(OutputStream.java:75)
	at hudson.remoting.ChunkedOutputStream.sendFrame(ChunkedOutputStream.java:93)
	at hudson.remoting.ChunkedOutputStream.drain(ChunkedOutputStream.java:89)
	at hudson.remoting.ChunkedOutputStream.write(ChunkedOutputStream.java:58)
	at java.io.OutputStream.write(OutputStream.java:75)
	at hudson.remoting.ChunkedCommandTransport.writeBlock(ChunkedCommandTransport.java:45)
	at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.write(AbstractSynchronousByteArrayCommandTransport.java:45)
	at hudson.remoting.Channel.send(Channel.java:582)
	at hudson.remoting.Channel.close(Channel.java:1156)
	at hudson.slaves.ChannelPinger$1.onDead(ChannelPinger.java:118)
	at hudson.remoting.PingThread.run(PingThread.java:96)
Ouch:
hudson.remoting.RequestAbortedException: java.io.IOException: Pipe closed
	at hudson.remoting.Request.abort(Request.java:297)
	at hudson.remoting.Channel.terminate(Channel.java:847)
	at hudson.remoting.Channel.close(Channel.java:1160)
	at hudson.slaves.ChannelPinger$1.onDead(ChannelPinger.java:118)
	at hudson.remoting.PingThread.run(PingThread.java:96)
	at ......remote call to ITKAWSWindows (i-0234e203d06253ae7)(Native Method)
	at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1416)
	at hudson.remoting.Request.call(Request.java:172)
	at hudson.remoting.Channel.call(Channel.java:780)
	at org.jenkinsci.modules.slave_installer.impl.ComputerListenerImpl.onOnline(ComputerListenerImpl.java:32)
	at hudson.slaves.SlaveComputer.setChannel(SlaveComputer.java:573)
	at hudson.slaves.SlaveComputer.setChannel(SlaveComputer.java:381)
	at hudson.plugins.ec2.win.EC2WindowsLauncher.launch(EC2WindowsLauncher.java:68)
	at hudson.plugins.ec2.EC2ComputerLauncher.launch(EC2ComputerLauncher.java:100)
	at hudson.slaves.SlaveComputer$1.call(SlaveComputer.java:253)
	at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Pipe closed
	at java.io.PipedInputStream.checkStateForReceive(PipedInputStream.java:260)
	at java.io.PipedInputStream.receive(PipedInputStream.java:226)
	at java.io.PipedOutputStream.write(PipedOutputStream.java:149)
	at java.io.OutputStream.write(OutputStream.java:75)
	at hudson.remoting.ChunkedOutputStream.sendFrame(ChunkedOutputStream.java:93)
	at hudson.remoting.ChunkedOutputStream.drain(ChunkedOutputStream.java:89)
	at hudson.remoting.ChunkedOutputStream.write(ChunkedOutputStream.java:58)
	at java.io.OutputStream.write(OutputStream.java:75)
	at hudson.remoting.ChunkedCommandTransport.writeBlock(ChunkedCommandTransport.java:45)
	at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.write(AbstractSynchronousByteArrayCommandTransport.java:45)
	at hudson.remoting.Channel.send(Channel.java:582)
	at hudson.remoting.Channel.close(Channel.java:1156)
	at hudson.slaves.ChannelPinger$1.onDead(ChannelPinger.java:118)
	at hudson.remoting.PingThread.run(PingThread.java:96)
ERROR: Connection terminated

A successful connection occur if the initialization is cancelled, then I manually launch the slave agent via the Jenkins GUI.
Add Comment Add Comment
 
This message was sent by Atlassian JIRA (v6.4.2#64017-sha1:e244265)
Atlassian logo

matt.mccormick@kitware.com (JIRA)

unread,
Apr 5, 2016, 8:48:01 PM4/5/16
to jenkinsc...@googlegroups.com

bstewart@novetta.com (JIRA)

unread,
May 27, 2016, 2:39:02 PM5/27/16
to jenkinsc...@googlegroups.com

I'm seeing this also, occurs frequently enough that i can't get my builds completed. I'm using the aws cli s3 for copying files to my slaves (as a powershell build step) when this occurs.

14:25:27 Agent went offline during the build
14:28:45 ERROR: Connection was broken: java.io.IOException: Pipe closed
14:28:45 at java.io.PipedInputStream.checkStateForReceive(Unknown Source)
14:28:45 at java.io.PipedInputStream.receive(Unknown Source)
14:28:45 at java.io.PipedOutputStream.write(Unknown Source)
14:28:45 at java.io.OutputStream.write(Unknown Source)
14:28:45 at hudson.remoting.ChunkedOutputStream.sendFrame(ChunkedOutputStream.java:93)
14:28:45 at hudson.remoting.ChunkedOutputStream.drain(ChunkedOutputStream.java:89)
14:28:45 at hudson.remoting.ChunkedOutputStream.write(ChunkedOutputStream.java:58)
14:28:45 at java.io.OutputStream.write(Unknown Source)
14:28:45 at hudson.remoting.ChunkedCommandTransport.writeBlock(ChunkedCommandTransport.java:45)
14:28:45 at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.write(AbstractSynchronousByteArrayCommandTransport.java:45)
14:28:45 at hudson.remoting.Channel.send(Channel.java:582)
14:28:45 at hudson.remoting.Channel.close(Channel.java:1156)
14:28:45 at hudson.slaves.ChannelPinger$1.onDead(ChannelPinger.java:119)
14:28:45 at hudson.remoting.PingThread.run(PingThread.java:96)
14:28:45

francisu@gmail.com (JIRA)

unread,
May 27, 2016, 2:47:01 PM5/27/16
to jenkinsc...@googlegroups.com

Can you make this happen on 1.33 (the current release)? There have been many fixes since 1.31.

bstewart@novetta.com (JIRA)

unread,
May 27, 2016, 2:50:02 PM5/27/16
to jenkinsc...@googlegroups.com

Correct, i'm getting it right now with 1.33

bstewart@novetta.com (JIRA)

unread,
May 27, 2016, 3:19:02 PM5/27/16
to jenkinsc...@googlegroups.com

Here is more:

ERROR: Connection terminated
java.io.IOException: Pipe closed

at java.io.PipedInputStream.read(Unknown Source)
at hudson.remoting.FlightRecorderInputStream.read(FlightRecorderInputStream.java:82)
at hudson.remoting.ChunkedInputStream.readHeader(ChunkedInputStream.java:73)
at hudson.remoting.ChunkedInputStream.readUntilBreak(ChunkedInputStream.java:103)
at hudson.remoting.ChunkedCommandTransport.readBlock(ChunkedCommandTransport.java:39)
at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:34)
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:48)
ERROR: Failed to install restarter
Ouch:
Connection terminated
hudson.remoting.RequestAbortedException: hudson.remoting.Channel$OrderlyShutdown
at hudson.remoting.Request.abort(Request.java:303)
at hudson.remoting.Channel.terminate(Channel.java:847)
at hudson.remoting.Channel$CloseCommand.execute(Channel.java:1080)
at hudson.remoting.Channel$1.handle(Channel.java:501)
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:60)
at ......remote call to Jenkins Slave (i-0f31fdf7)(Native Method)


at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1416)
at hudson.remoting.Request.call(Request.java:172)
at hudson.remoting.Channel.call(Channel.java:780)

at org.jenkinsci.modules.systemd_slave_installer.SlaveInstallerFactoryImpl.createIfApplicable(SlaveInstallerFactoryImpl.java:33)
at org.jenkinsci.modules.slave_installer.SlaveInstallerFactory.createIfApplicable(SlaveInstallerFactory.java:29)
at org.jenkinsci.modules.slave_installer.SlaveInstallerFactory.createFor(SlaveInstallerFactory.java:46)
at org.jenkinsci.modules.slave_installer.impl.ComputerListenerImpl.onOnline(ComputerListenerImpl.java:30)


at hudson.slaves.SlaveComputer.setChannel(SlaveComputer.java:573)
at hudson.slaves.SlaveComputer.setChannel(SlaveComputer.java:381)

at hudson.plugins.ec2.win.EC2WindowsLauncher.launch(EC2WindowsLauncher.java:69)
at hudson.plugins.ec2.EC2ComputerLauncher.launch(EC2ComputerLauncher.java:122)


at hudson.slaves.SlaveComputer$1.call(SlaveComputer.java:253)
at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)

at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: hudson.remoting.Channel$OrderlyShutdown
at hudson.remoting.Channel$CloseCommand.execute(Channel.java:1080)
at hudson.remoting.Channel$1.handle(Channel.java:501)
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:60)
Caused by: Command close created at
at hudson.remoting.Command.<init>(Command.java:56)
at hudson.remoting.Channel$CloseCommand.<init>(Channel.java:1074)
at hudson.remoting.Channel$CloseCommand.<init>(Channel.java:1072)
at hudson.remoting.Channel.close(Channel.java:1156)
at hudson.remoting.Channel.close(Channel.java:1138)
at hudson.remoting.Channel$CloseCommand.execute(Channel.java:1079)
... 2 more
hudson.remoting.RequestAbortedException: hudson.remoting.Channel$OrderlyShutdown
at hudson.remoting.Request.abort(Request.java:303)
at hudson.remoting.Channel.terminate(Channel.java:847)
at hudson.remoting.Channel$CloseCommand.execute(Channel.java:1080)
at hudson.remoting.Channel$1.handle(Channel.java:501)
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:60)
at ......remote call to Jenkins Slave (i-0f31fdf7)(Native Method)


at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1416)
at hudson.remoting.Request.call(Request.java:172)
at hudson.remoting.Channel.call(Channel.java:780)

at jenkins.slaves.restarter.JnlpSlaveRestarterInstaller.install(JnlpSlaveRestarterInstaller.java:52)
at jenkins.slaves.restarter.JnlpSlaveRestarterInstaller.access$000(JnlpSlaveRestarterInstaller.java:33)
at jenkins.slaves.restarter.JnlpSlaveRestarterInstaller$1.call(JnlpSlaveRestarterInstaller.java:39)
at jenkins.slaves.restarter.JnlpSlaveRestarterInstaller$1.call(JnlpSlaveRestarterInstaller.java:36)
at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: hudson.remoting.Channel$OrderlyShutdown
at hudson.remoting.Channel$CloseCommand.execute(Channel.java:1080)
at hudson.remoting.Channel$1.handle(Channel.java:501)
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:60)
Caused by: Command close created at
at hudson.remoting.Command.<init>(Command.java:56)
at hudson.remoting.Channel$CloseCommand.<init>(Channel.java:1074)
at hudson.remoting.Channel$CloseCommand.<init>(Channel.java:1072)
at hudson.remoting.Channel.close(Channel.java:1156)
at hudson.remoting.Channel.close(Channel.java:1138)
at hudson.remoting.Channel$CloseCommand.execute(Channel.java:1079)
... 2 more

Help us localize this page Page generated: May 27, 2016 3:16:24 PM EDTREST APIJenkins ver. 2.6

bitwiseman@gmail.com (JIRA)

unread,
May 13, 2019, 10:43:03 AM5/13/19
to jenkinsc...@googlegroups.com
Liam Newman resolved as Fixed
 
Change By: Liam Newman
Status: Open Resolved
Resolution: Fixed
This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d)

bitwiseman@gmail.com (JIRA)

unread,
May 13, 2019, 10:43:04 AM5/13/19
to jenkinsc...@googlegroups.com
Liam Newman commented on Bug JENKINS-34044
 
Re: EC2 Plugin: Windows: java.io.IOException: Pipe closed

I believe this is fixed in Jenkins v2.177.
Please comment if this should be reopened.

Pierson_Yieh@intuit.com (JIRA)

unread,
Feb 27, 2020, 6:20:03 PM2/27/20
to jenkinsc...@googlegroups.com

Requesting this to be reopened. We're seeing this behavior in our Jenkins builds (where the Jenkins Master will lose/close the connection to the Windows agent nodes at the beginning or in the middle of a build). 
We're using the latest release of the ec2-plugin 1.49.1 and Jenkins version 2.204.2 
This message was sent by Atlassian Jira (v7.13.12#713012-sha1:6e07c38)
Atlassian logo

Pierson_Yieh@intuit.com (JIRA)

unread,
Feb 27, 2020, 6:22:03 PM2/27/20
to jenkinsc...@googlegroups.com
Pierson Yieh edited a comment on Bug JENKINS-34044
Requesting this to be reopened. We're seeing this behavior in our Jenkins builds (where the Jenkins Master will lose/close the connection to the Windows agent nodes at the beginning or in the middle of a build). 
We're using the latest release of the ec2-plugin 1.49.1 and Jenkins version 2.204.2 


CC: [~narayanan]

Pierson_Yieh@intuit.com (JIRA)

unread,
Feb 27, 2020, 6:23:03 PM2/27/20
to jenkinsc...@googlegroups.com
Pierson Yieh edited a comment on Bug JENKINS-34044
Requesting this to be reopened. We're seeing this behavior in our Jenkins builds (where the Jenkins Master will lose/close the connection to the Windows agent nodes at the beginning or in the middle of a build). 
We're using the latest release of the ec2-plugin 1.49.1 and Jenkins version 2.204.2 
We're happy to contribute a fix, but could use some assistance in pointing us in the right direction of where to start looking. 

 CC: [~narayanan]

Pierson_Yieh@intuit.com (JIRA)

unread,
Feb 28, 2020, 7:05:06 PM2/28/20
to jenkinsc...@googlegroups.com
Change By: Pierson Yieh
Resolution: Fixed
Status: Resolved In Review

pipeline-security-members@cloudbees.com (JIRA)

unread,
Mar 3, 2020, 12:05:07 PM3/3/20
to jenkinsc...@googlegroups.com
CloudBees Pipeline Security commented on Bug JENKINS-34044
 
Re: EC2 Plugin: Windows: java.io.IOException: Pipe closed

Pierson Yieh
This issue was specific to when nodes come online. If they are already online (running a build) and then get disconnected, that not the same as this issue.
It sounds to me like you'll want to open a new issue. Feel free to tag me and I'll see if I can find time to take a look. You'll definitely want to include the callstack from your log when the error occurs.

pipeline-security-members@cloudbees.com (JIRA)

unread,
Mar 3, 2020, 12:05:10 PM3/3/20
to jenkinsc...@googlegroups.com
Change By: CloudBees Pipeline Security
Status: In Review Resolved
Resolution: Fixed

pipeline-security-members@cloudbees.com (JIRA)

unread,
Mar 3, 2020, 12:07:04 PM3/3/20
to jenkinsc...@googlegroups.com
CloudBees Pipeline Security updated an issue
Comment: [~pyieh]

This issue was specific to when nodes come online.  If they are already online (running a build) and then get disconnected, that not the same as this issue.
It sounds to me like you'll want to open a new issue. Feel free to tag me and I'll see if I can find time to take a look.  You'll definitely want to include the callstack from your log when the error occurs.

bitwiseman@gmail.com (JIRA)

unread,
Mar 3, 2020, 12:07:05 PM3/3/20
to jenkinsc...@googlegroups.com
Change By: Liam Newman
Resolution: Fixed
Status: Resolved In Review

bitwiseman@gmail.com (JIRA)

unread,
Mar 3, 2020, 12:07:08 PM3/3/20
to jenkinsc...@googlegroups.com
Change By: Liam Newman
Status: In Review Resolved
Resolution: Fixed

bitwiseman@gmail.com (JIRA)

unread,
Mar 3, 2020, 12:07:11 PM3/3/20
to jenkinsc...@googlegroups.com


This issue was specific to when nodes come online. If they are already online (running a build) and then get disconnected, that not the same as this issue.
It sounds to me like you'll want to open a new issue. Feel free to tag me and I'll see if I can find time to take a look. You'll definitely want to include the callstack from your log when the error occurs.

Reply all
Reply to author
Forward
0 new messages