[JIRA] (JENKINS-60987) How to terminate Job container with non-0 gracePeriodSeconds

[cofyc.jackson@gmail.com (JIRA)]

Yecheng Fu created an issue

Jenkins / JENKINS-60987 How to terminate Job container with non-0 gracePeriodSeconds Issue Type: Bug Assignee: Unassigned Components: kubernetes-plugin Created: 2020-02-06 05:15 Environment: Kubernetes 1.12.8 Kubernetes Plugin 1.19.3 Priority: Minor Reporter: Yecheng Fu hi,  I'm running CI with the Jenkins Kubernetes plugin. In the job container, an internal k8s cluster is started by kind. So lots of containers are started in the container.  I'm trying to clean contains on termination (SIGTERM) to avoid cgroup leaking, but found that the pod is terminated with 0 grace period seconds. So it's not possible for us to clean stale containers when the job is aborted. Kubelet logs: Feb 06 12:28:34 172.16.5.67 kubelet[15125]: I0206 12:28:34.209179 15125 kubelet.go:1899] SyncLoop (DELETE, "api"): "build-tidb-operator-master-kind-249-9b2v8-gz33b-2krwn_jenkins-ci(d654cf7e-4893-11ea-b4a3-1866da9d20ee)" Feb 06 12:28:34 172.16.5.67 kubelet[15125]: I0206 12:28:34.209712 15125 kuberuntime_container.go:555] Killing container "docker://280bf48b2cda6f0dcc2bb4a6ac317913962054d70a00ed974bfe280df78b1719" with 0 second grace period Feb 06 12:28:34 172.16.5.67 kubelet[15125]: I0206 12:28:34.209802 15125 kuberuntime_container.go:555] Killing container "docker://b38a77770d867e24ecd606fc4a9391763c35553b61de35f7f8d013d49c7c6a0a" with 0 second grace period Feb 06 12:28:34 172.16.5.67 kubelet[15125]: I0206 12:28:34.211965 15125 kubelet.go:1893] SyncLoop (REMOVE, "api"): "build-tidb-operator-master-kind-249-9b2v8-gz33b-2krwn_jenkins-ci(d654cf7e-4893-11ea-b4a3-1866da9d20ee)" Feb 06 12:28:34 172.16.5.67 kubelet[15125]: I0206 12:28:34.234719 15125 kubelet_pods.go:1083] Killing unwanted pod "build-tidb-operator-master-kind-249-9b2v8-gz33b-2krwn" Feb 06 12:28:34 172.16.5.67 kubelet[15125]: I0206 12:28:34.234785 15125 kuberuntime_container.go:555] Killing container "docker://280bf48b2cda6f0dcc2bb4a6ac317913962054d70a00ed974bfe280df78b1719" with 0 second grace period Feb 06 12:28:34 172.16.5.67 kubelet[15125]: I0206 12:28:34.234794 15125 kuberuntime_container.go:555] Killing container "docker://b38a77770d867e24ecd606fc4a9391763c35553b61de35f7f8d013d49c7c6a0a" with 0 second grace period Note that job pod spec has `TerminationGracePeriodSeconds` set to 30, so I guess in deletion, `metadata.DeletionGracePeriodSeconds` is set to 0.  Do you know how to clean stale resources when the job is aborted? Can Kubernetes plugin be configured to abort the pod with non-zero DeletionGracePeriodSeconds? Add Comment

This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f)