Deprecating and removing Restore via CreateContainer in containerd
Samuel Karp
Hello,
I wanted to send a follow-up email from yesterday’s discussion at the SIG Node Weekly meeting.
KEP-2008 (Forensic Container Checkpointing) introduced a CRI API for producing checkpoints of containers but did not specify a mechanism for restoring those checkpoints. At the same time, a proof-of-concept restoration path via CreateContainer was first implemented in CRI-O and later in containerd. This restoration path operates implicitly; the runtime determines a given image: in the pod spec is a checkpoint, and automatically performs a restore instead of a standard container creation.
Container restore has been implicated in several recent CVEs in containerd. While these have been patched, the implicit behavior means Kubernetes has no visibility of the restore or policy levers to control whether restore is allowed. This also means there is no easy way to detect whether this feature was used by any running Pods in the cluster through the Kubernetes API. KEP-5823 (Pod-Level Checkpoint/Restore) adds an explicit API for RestorePod.
The containerd project has decided to deprecate and remove this codepath in favor of the explicit RestorePod API currently moving through the KEP process. This restores containerd to specified CRI CreateContainer behavior rather than implicitly (and invisibly) restoring from checkpoints. containerd 2.3.4 (LTS, EOL April 30, 2028) and 2.2.7 (EOL November 6, 2026) both retain the feature, but disable it by default and require a new enable_experimental_restore_via_create configuration option to explicitly opt-in to this behavior. containerd 2.4 (tentatively expected for release at the end of this month) will not support restore via CreateContainer. Cluster administrators who are unsure of whether this feature is used can enable the enable_experimental_restore_via_create option and watch for the io.containerd.deprecation/cri-create-container-checkpoint-restore deprecation warning. Cluster administrators can also look for messages like the following in the containerd log: “Found checkpoint of container <name> in <ref>”, “Prepared <id> for restore”, and “Restored container <id> successfully”.
If cluster administrators plan to keep restore via CreateContainer enabled, the containerd project cautions that only trusted users should be granted permission to create pods and that only trusted values should be specified in the image: field of a pod.
On behalf of the containerd project and SIG Node,
Samuel Karp