cgroup v1 deprecation

[Kevin Hannon]

Hello,

The latest trends in the linux community is to phase out cgroup v1.
Systemd 258 has announced the removal of cgroup v1 code so I propose
to deprecate cgroup v1 as stage 1 and in 1.38 we move to remove cgroup
v1 from the code base.

I drafted https://github.com/kubernetes/enhancements/issues/5573.

If you have concerns, please let sig-node know and comment on the KEP or issue.

Kevin Hannon

[Kevin Hannon]

I pushed an update to the KEP where we removed the time when we push for removal.

The KEP proposes moving failCgroupv1 from false to true and clearly stating that cgroup v1 is deprecated. If someone is still using cgroup v1 they can turn off that flag.

We will still support testing on cgroup v1 but we can clearly state that we have plans to remove in the future.

I am not aware of any cgroup v1 bugs we have receive since we put cgroup v1 in maintenance mode.  

Kevin
On Friday, October 3, 2025, Benjamin Elder <benth...@google.com> wrote:

We release a lot of things with testing gaps ... I think it would be inconsistent to rush to remove for that reason at the moment.

I investigated this a bit yesterday, and it turns out that the current SIG Node e2e release blocking signal is not replicated to any of the release branches, not even 1.34.
So we're clearly releasing cgroup v2 with coverage gaps as well for any backports.

How often are we actually seeing cgroupv1-specific bugs, with it in maintenance mode?

I reached out to #containerd-dev in the CNCF slack and AFAICT there are no plans to drop cgroup v1 there.

The kernel, runc, and docker don't seem to either, I think the tangents about kernels have gotten a bit off-track, what you need is an *init* that still works, the kernel is not dropping support.

On Fri, Oct 3, 2025 at 9:04 AM Kevin Hannon <keha...@redhat.com> wrote:

So to me, the main problem is maintaining test infrastructure for cgroup v1 and how much effort we put in these jobs to keep running.

If there is a failure on a cgroup v1 related node, the interest in fixing this from the linux community is pretty low.
I was motivated to bring up cgroup v1 deprecation because of https://github.com/kubernetes/kubernetes/issues/134142#issuecomment-3325033801.

We could probably more easily support this if we use something like AlmaLinux 8 or Rocky 8 in our CI. Maybe AlmaLinux 9 or Rocky 9 would be fine as cgroup v1 is deprecated in those distributions but not yet removed. 

Benjamin Elder mentioned that we already use AlmaLinux 8 in some of our kind tests.

So I guess that could be an option if we still want to support cgroup v1 until the linux distributions fully remove it.

On Fri, Oct 3, 2025 at 11:25 AM Tim Hockin <tho...@google.com> wrote:

I agree with Dan's sentiment, but what I want to understand most is if there's a real reason to remove code from Kubernetes, or if it's just an opportunistic cleanup. 

Don't get me wrong, I love deleting old unused unmaintained code.  The question is not whether it's worth doing, the question is when is it worth doing?  If we leave that code in tree for another year or even 3 years, do bad things happen? I honestly don't know that area of the code very well.

I know there are testing problems, and I know that every time we add a new feature or fix a bug, we have to decide whether to fix it in the old code or not.  Are there other considerations?

If the cost of leaving it for a while is not high, my preference is to leave it and let the usage drain as naturally as possible.  If it's actually a substantial problem to keep, then we can talk about how to ease the pain for as many customers as possible who are still using V1.

Tim

On Fri, Oct 3, 2025, 7:53 AM Dan Winship <danwi...@redhat.com> wrote:

On 10/2/25 2:24 PM, 'Alvaro Aleman' via dev wrote:
> As an anecdotal example, Amazon Linux 2 does not support cgroup v2 and
> is supported until 2026-06-30, <https://aws.amazon.com/amazon-linux-2/
> faqs/#topic-0> I would expect it to still see a decent bit of usage on AWS

Enterprise distros are "supported" for a long time, but after a while
there's an assumption that you're supporting them in running their
existing workloads, not supporting them in adopting new bleeding-edge
software...

-- Dan

>
> On Thu, 2 Oct 2025 at 14:14, 'Benjamin Elder' via dev <d...@kubernetes.io
> <mailto:d...@kubernetes.io>> wrote:
>
>     Thanks, also I want to note that I'm playing devil's advocate
>     here ... KIND also gets simpler when we drop v1 so I understand the
>     impetus, but I'm wary of triggering dockershim style blowback.
>
>     We should probably be pretty confident that impact will be low
>     before any hard cutoff.
>
>     On Thu, Oct 2, 2025 at 10:36 AM Kevin Hannon <keha...@redhat.com
>     <mailto:keha...@redhat.com>> wrote:
>
>         I pushed a commit adding my research on linux
>         distributions. https://github.com/kubernetes/enhancements/
>         pull/5574/commits/d27a68bd959af18947f493b25ba10aa8e938bf08
>         <https://github.com/kubernetes/enhancements/pull/5574/commits/
>         d27a68bd959af18947f493b25ba10aa8e938bf08>
>
>         On Thu, Oct 2, 2025 at 1:14 PM 'Benjamin Elder' via dev

>         <d...@kubernetes.io <mailto:d...@kubernetes.io>> wrote:
>
>             > Systemd 258 has announced the removal of cgroup v1 code so
>             I propose
>

>             AIUI, the ecosystem has offerings that don't use systemd at
>             all (e.g. talos?), I don't think we should necessarily be
>             deciding based on systemd.
>
>             > The main reason to remove it is to avoid maintaining and
>             testing it going forward. Whether some distros support it or
>             not isn't relevant until we are sure that all of our needs
>             are completely met by V2.  
>
>             Testing is pretty cheap in this respect, we already run CI
>             for multiple container runtimes and versions and we don't
>             need a 5k node scale cluster to test cgroups.
>
>             Maintenance might be less, but if we're not adding new
>             features to this code already ...
>
>             > So my question is what is preventing our users from
>             adopting V2?
>
>             1) If you're running minikube --vm-driver=docker (or
>             podman), kind, k3d, etc. then you're using whatever the host
>             distro provides. So the developer distros at minimum are
>             relevant.
>
>             2) We instituted a breaking change in cgroupv2 to OOMKill
>             group instead of individual processes, there's a kubelet
>             config option now, but that requires access to configure
>             kubelet, versus sticking to cgroupv1 nodes.
>             ... This has caused a lot of problems <https://github.com/
>             kubernetes-sigs/prow/issues/210> for the project's own CI ... 
>
>             3) Again, distro defaults. I doubt many are tweaking kernel
>             boot params on their nodes.
>
>
>             On Thu, Oct 2, 2025 at 7:49 AM 'Tim Hockin' via dev
>             <d...@kubernetes.io <mailto:d...@kubernetes.io>> wrote:
>
>                 The main reason to remove it is to avoid maintaining and
>                 testing it going forward. Whether some distros support
>                 it or not isn't relevant until we are sure that all of
>                 our needs are completely met by V2.  
>
>                 So my question is what is preventing our users from
>                 adopting V2?
>
>                 On Thu, Oct 2, 2025, 7:03 AM Sean McGinnis
>                 <sean.m...@gmail.com
>                 <mailto:sean.mcginnis@gmail.com>> wrote:
>
>                     I'd love to hear from some distro folks on current
>                     cgroup support plans.
>
>                     If cgroupv1 is just discouraged in all current
>                     releases, that's one thing. But if it is actively
>                     being removed from upcoming releases, that raises
>                     the priority for Kubernetes to start strongly
>                     avoiding (and supporting) v1 support.
>
>                     Of course it's also tricky on the other end of the
>                     spectrum. If there are other currently "supported"
>                     releases out there that default to v1, that will
>                     also cause friction. But it would be good to
>                     understand the scope of that so we can try to move
>                     forward with the least amount of pain for users.
>
>                     Sean
>
>                     On Thursday, October 2, 2025 at 7:59:36 AM UTC-5

>                     Kevin Hannon wrote:
>
>                         Hello,
>
>                         The latest trends in the linux community is to
>                         phase out cgroup v1.
>                         Systemd 258 has announced the removal of cgroup
>                         v1 code so I propose
>                         to deprecate cgroup v1 as stage 1 and in 1.38 we
>                         move to remove cgroup
>                         v1 from the code base.
>
>                         I drafted https://github.com/kubernetes/

>                         enhancements/issues/5573 <https://github.com/
>                         kubernetes/enhancements/issues/5573>.

>
>                         If you have concerns, please let sig-node know
>                         and comment on the KEP or issue.
>
>                         Kevin Hannon
>

>                     --
>                     You received this message because you are subscribed
>                     to the Google Groups "dev" group.
>                     To unsubscribe from this group and stop receiving
>                     emails from it, send an email to
>                     dev+uns...@kubernetes.io
>                     <mailto:dev+unsubscribe@kubernetes.io>.
>                     To view this discussion visit https://
>                     groups.google.com/a/kubernetes.io/d/msgid/
>                     dev/403b3a67-67bf-40da-
>                     b4b5-2b5f960ad37fn%40kubernetes.io <https://
>                     groups.google.com/a/kubernetes.io/d/msgid/
>                     dev/403b3a67-67bf-40da-
>                     b4b5-2b5f960ad37fn%40kubernetes.io?
>                     utm_medium=email&utm_source=footer>.
>
>                 --
>                 You received this message because you are subscribed to
>                 the Google Groups "dev" group.
>                 To unsubscribe from this group and stop receiving emails
>                 from it, send an email to dev+uns...@kubernetes.io
>                 <mailto:dev+unsubscribe@kubernetes.io>.
>                 To view this discussion visit https://groups.google.com/
>                 a/kubernetes.io/d/msgid/dev/
>                 CAO_RewY2MM%3Df9Rc17qQruirbDpO_X9TgygGMj%2B%2Ba0q%3Dj9aCb9A%40mail.gmail.com <https://groups.google.com/a/kubernetes.io/d/msgid/dev/CAO_RewY2MM%3Df9Rc17qQruirbDpO_X9TgygGMj%2B%2Ba0q%3Dj9aCb9A%40mail.gmail.com?utm_medium=email&utm_source=footer>.
>
>             --
>             You received this message because you are subscribed to the
>             Google Groups "dev" group.
>             To unsubscribe from this group and stop receiving emails
>             from it, send an email to dev+uns...@kubernetes.io
>             <mailto:dev+unsubscribe@kubernetes.io>.
>             To view this discussion visit https://groups.google.com/a/
>             kubernetes.io/d/msgid/dev/
>             CAOZRXm9q%2BpWjmpBRYPDb1kOTYZHDMjz%3D5hfjUZ_AH23J4WSzOA%40mail.gmail.com <https://groups.google.com/a/kubernetes.io/d/msgid/dev/CAOZRXm9q%2BpWjmpBRYPDb1kOTYZHDMjz%3D5hfjUZ_AH23J4WSzOA%40mail.gmail.com?utm_medium=email&utm_source=footer>.
>
>     --
>     You received this message because you are subscribed to the Google
>     Groups "dev" group.
>     To unsubscribe from this group and stop receiving emails from it,
>     send an email to dev+uns...@kubernetes.io
>     <mailto:dev+unsubscribe@kubernetes.io>.
>     To view this discussion visit https://groups.google.com/a/
>     kubernetes.io/d/msgid/dev/CAOZRXm-
>     %3DwWHjALAwagEcKpLjfi%3Dn26YRHZUyPJXym0jRF3OkPQ%40mail.gmail.com
>     <https://groups.google.com/a/kubernetes.io/d/msgid/dev/CAOZRXm-
>     %3DwWHjALAwagEcKpLjfi%3Dn26YRHZUyPJXym0jRF3OkPQ%40mail.gmail.com?
>     utm_medium=email&utm_source=footer>.
>
> --
> You received this message because you are subscribed to the Google
> Groups "dev" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to dev+uns...@kubernetes.io
> <mailto:dev+unsubscribe@kubernetes.io>.
> To view this discussion visit https://groups.google.com/a/kubernetes.io/
> d/msgid/dev/
> CAAkVc2oqfuEcOJPOwrA9kp8%2BC6cjzQPfnVxMFKGv6RdcVzvrAQ%40mail.gmail.com
> <https://groups.google.com/a/kubernetes.io/d/msgid/dev/
> CAAkVc2oqfuEcOJPOwrA9kp8%2BC6cjzQPfnVxMFKGv6RdcVzvrAQ%40mail.gmail.com?
> utm_medium=email&utm_source=footer>.