How do I check Job failure reason?

1,761 views
Skip to first unread message

omega...@gmail.com

unread,
Jun 24, 2018, 3:45:47 PM6/24/18
to Kubernetes user discussion and Q&A
I have a kubernetes Job that has a backOffLimit set as 2, restartPolicy set as Never. However, I've seen situations where some jobs get evicted twice and finally completed and some certain job evicted twice but stay failed. I'm wondering how I can further debug this? From the source code it seems there's a job failureReason being sent out: https://github.com/kubernetes/kubernetes/blob/34759c2dfb2e65e5b15e278dcdb785140b2c8d5f/pkg/controller/job/job_controller.go#L510.

The only relevant output I can see from kubectl describe job is:

Start Time: Sat, 23 Jun 2018 05:33:19 -0700
Pods Statuses: 0 Running / 1 Succeeded / 2 Failed

and

Start Time: Sat, 23 Jun 2018 05:33:20 -0700
Pods Statuses: 0 Running / 0 Succeeded / 2 Failed

with the events field being <none>

Thanks in advance.

Timo Reimann

unread,
Jun 25, 2018, 2:09:20 AM6/25/18
to Kubernetes user discussion and Q&A
Have you possibly tried "kubectl get job -o yaml" to check if that output shows the failure reason?

Siyang Yu

unread,
Jun 25, 2018, 2:42:08 PM6/25/18
to kubernet...@googlegroups.com
Hi Timo,

Yeah I saw the failure reason as 'BackoffLimitExceeded'. However, I would expect the the pod to be retried another time as the other completed jobs did since I set the backoffLimit as 2. Is there anything could cause this discrepancy?

Thanks

2018-06-24 23:09 GMT-07:00 'Timo Reimann' via Kubernetes user discussion and Q&A <kubernet...@googlegroups.com>:
Have you possibly tried "kubectl get job -o yaml" to check if that output shows the failure reason?

--
You received this message because you are subscribed to a topic in the Google Groups "Kubernetes user discussion and Q&A" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/kubernetes-users/BzbGsjkYOvw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to kubernetes-users+unsubscribe@googlegroups.com.
To post to this group, send email to kubernetes-users@googlegroups.com.
Visit this group at https://groups.google.com/group/kubernetes-users.
For more options, visit https://groups.google.com/d/optout.

Reply all
Reply to author
Forward
0 new messages