| I see the same thing. We have podRetention set to never, but out of the probably hundreds of jobs we run per day, a handful of the pods stick around in an error state or in "Running" where only a subset of the containers are running (i.e. READY: 3/4 STATUS: Running). FWIW, our jobs don't reuse pods. I suppose if they did we might ( ? ) see this less often. Cleanup is not terribly onerous except that it has to be done every day. I have a hard time seeing how this can be classified as an enhancement rather than a bug. Given long enough, this will cause the system to stop working because of resource exhaustion. We happen to be on kubernetes 1.12.10, plugin version 1.18.3, and Jenkins version 2.176.3, but we also seen this with slightly older versions of all three, and I believe in testing with newer versions of Jenkins and plugin (but we're not running enough jobs on those newer versions to see it reliably). |
