Jenkins pipeline hangs when pod deleted

[Miquel Catany]

Hi,

We are using GCP preemtible instances to deploy pod agents with Kubernetes Plugin. We want to be able to retry the pipeline if the node/pod is evicted, but our main problem is that when the pipeline is executing sh commands and the node/pod is evicted/deleted,  the pipeline instead of failing/aborted is hung undefinitely. This problem was supposed to be fixed in https://issues.jenkins.io/browse/JENKINS-49707 and https://issues.jenkins.io/browse/JENKINS-59340 but it doesn't seem to work in our use case. The only solution to this is to add a timeout (which is useless for us) or abort it manually. Any insights on this?

I attach our current Jenkins file and the job results:

Thanks,

Miquel

[Miquel Catany]

Cannot contact ci-worker-dind-jsvbh: hudson.remoting.ChannelClosedException: Channel "hudson.remoting.Channel@13b5e548:JNLP4-connect connection from 10.251.13.10/10.251.13.10:48846": Remote call on JNLP4-connect connection from 10.251.13.10/10.251.13.10:48846 failed. The channel is closing down or has closed down

[Miquel Catany]

```

def worker_pipeline(){

 try{

   node("xxxxxxxx") {

      stage('Checkout SCM'){

      git branch: 'xxxx',

        credentialsId: 'xxx',

        url: 'https://github.com/xxxxx/xxxx.git'

      }

     container('xxxxxxxx'){

      stage('Run Builds'){

       "base_image": {

         sh 'gcloud builds submit --config .ci/builds/base_image_cloudbuild.yaml'

       },

       "assets_dependencies": {

         sh 'gcloud builds submit --config .ci/builds/assets_dependencies_cloudbuild.yaml'

       }

      }

     }

   }

 }

 catch(err)

 {

  if ( "${err}" == "hudson.remoting.RequestAbortedException: java.nio.channels.ClosedChannelException" ){

   echo "The Pipeline has failed because of a Connection fail, so retry"

   worker_pipeline()

  }  

  else{

   echo "${err}"

   echo "The pipeline has failed, exit"

   throw new Exception("Pipeline has failed, exit") 

  }

 }

}

// Call main function

worker_pipeline()

```