Question regarding the one shot workers to be gracefully terminated

35 views
Skip to first unread message

Victor Martinez

unread,
Jun 22, 2021, 12:28:29 PM6/22/21
to Jenkins Developers
Hi all,

When using the Google Compute Engine plugin with `oneShot` workers, I see some stacktraces with the below error message.

2021-06-22 16:09:45.264+0000 [id=497] INFO o.j.p.workflow.job.WorkflowRun#finish: hello-world-gce #9 completed: SUCCESS

2021-06-22 16:09:45.936+0000 [id=497] INFO hudson.remoting.Request$2#run: Failed to send back a reply to the request hudson.remoting.Request$2@67760e47: hudson.remoting.ChannelClosedException: Channel "hudson.remoting.Channel@c9422c06:obs11-ubuntu-18-linux-14sl9x": channel is already closed

Therefore, the CloudProvisioningListener#onFailure is executed even though the build did finish successfully.

As far as I see the Google Compute Engine uses the OnceRetentionStrategy when the task has finished and therefore the taskCompleted is executed for the AbstractCloudSlave.

IIUC, the piece of stacktrace is caused by https://github.com/jenkinsci/jenkins/blob/862acf3e2d3fd48330a7326bd8901d82085244d6/core/src/main/java/hudson/slaves/NodeProvisioner.java#L235-L241

Question, what's the best approach to gracefully terminate the connection in the cloud providers? 

My aim, is to extend the CloudProvisioningListener class to monitor what cloud workers failed genuinely versus the ones that were killed gracefully?

Thanks



Jesse Glick

unread,
Jun 22, 2021, 1:13:46 PM6/22/21
to Jenkins Dev
On Tue, Jun 22, 2021 at 12:28 PM Victor Martinez <victormar...@gmail.com> wrote:

INFO hudson.remoting.Request$2#run: Failed to send back a reply to the request hudson.remoting.Request$2@…: hudson.remoting.ChannelClosedException: Channel "hudson.remoting.Channel@…": channel is already closed

If you can reproduce your warning, please try with https://github.com/jenkinsci/remoting/pull/463 to see what exactly the `Request` is and whether my `TODO` comment would help.

Victor Martinez

unread,
Jun 23, 2021, 4:42:26 PM6/23/21
to Jenkins Developers
Thanks Jesse, I'll prepare the context with that PR and reproduce the warning
Reply all
Reply to author
Forward
0 new messages