dockershim removal survey results
Sergey Kanzhelev
Here are the results of the end users survey asking about readiness for dockershim removal: https://kubernetes.io/blog/2021/11/12/are-you-ready-for-dockershim-removal/ You can leave comments on these results in this google doc: https://docs.google.com/document/d/1DiwCRJffBjoLuDguW9auMXS2NTmkp3NAS7SGJQBu8JY/edit#bookmark=id.w2f1vwb2pfv9
The objective for the survey was to raise awareness of the upcoming major change, assess the readiness, and understand issues blocking end users adoption of alternative runtimes.
Preliminary results were shared and discussed at contribex comms meeting: https://docs.google.com/document/d/1KDoqbw2A6W7rLSbIRuOlqH8gkoOnp2IHHuV9KyJDD2c/edit#bookmark=id.ri1bspz9t94o. We, as a broader k8s community, will need to keep helping users through this transition.
Survey will be open for more replies. Survey is quite active and since yesterday’s snapshot we already have 7 more replies.
Results
TL;DR; The number of people who don't feel prepared is over 40%. The most common reason for not migrating is a dependency on docker, in most cases those dependencies are from both - self-authored and third-party tools. 40% of respondents who do NOT feel ready for dockershim removal planned to adopt 1.24 in 2022. There are a list of tasks I compiled from free text comments. The need to re-educate personnel, change processes and tooling, without the direct business benefit (basically the cost of operation) is a big complaint that we also need to address as a community.
Do you know that dockershim support will be removed in k8s 1.24?
This is surprising that 17% weren't aware of the removal. This is one of the things discussed at sig contribex, and here https://github.com/kubernetes/community/issues/5344.
Do you feel you have enough information about your migration options and you prepared for the dockershim removal?
The 40% respondents saying they do not feel prepared is a big number that calls for extra analysis.
Do you know what container runtime do you use in your cluster(s)?
If you are still using dockershim for any of your clusters, what is the reason?
This is a multiple choice question. 31% named dependencies on docker a reason to stay with dockershim:
22% self-authored tools
26.9% - third party tools
How do you run Kubernetes?
Surprisingly, there were answers that hosting provider do not support alternative runtimes for hosting providers which do have it. This may indicate the lack of documentation or issues blocking the migration on a specific provider. It wasn’t clear from replies.
What version Kubernetes do you run?
The survey’s respondents are keeping up with k8s versions reasonably well. This adds the weight to the results. It is a known fact, that there are users who fail to even keep up with upgrades. Those customers would be most conservative with any changes k8s implements.
Out of respondents who do not run any 1.1x workloads (only run supported 1.20+), only 212 out of 349 (60%) feel prepared for dockershim removal.
When would you estimate you will adopt the version 1.24 of Kubernetes?
Out of all people who answered they are NOT ready (208):
25 - first half of 2022
59 - by end of 2022
25 - first half of 2023
30 - end of 2023
60 - later
9 - no answer
Again, like a previous question, shows that many survey respondents are running up to date k8s versions. From people who responded that they do NOT feel ready for dockershim removal, 40% was planning to adopt 1.24 next year.
Comments analysis
Quotes are posted as example of comments where they do not expose any PII. All comments will be shared with SIG Node chairs.
Blocking issues
There are blocking issue of this migration. Some are known, some - not. Not all comments explained specific issues:
Still waiting for base OS to sort out install for cri-o or containerd https://github.com/coreos/fedora-coreos-tracker/issues/767
“We had some container failures under containerd, so we rolled back.”
Problems with in-place migration steps
“Given the infrastructure stack we are on. We have not found a good alternative to docker. We have been considering it for some time, but have not found a good alternative yet.”
“I don't know what is the best option in our environment”
No time/resources
There were quite a few comments like this.
“Docker is the easiest tool to use currently, and I haven't had time to migrate.”
“Kubernetes frequently changes which means I constantly need to adapt to the changes, which gets in the way of my main job. Dockershim removal is yet another major task that I need to plan for on top of all the other Kubernetes changes.”
“Not at all looking forward to losing access to status and logs through docker command line client”
“I don't understand how to even check, let alone migrate between, container run times, in Kubernetes. When I searched several months ago I found no real community resources. I've got a lot to learn about k8s however I'm still considered generally informed compared to most other engineers I work with.”
“We are still using k8s v1.15, the migration project is not yet planned”
Documentation tasks
This is the list of documentation tasks we should consider to do to help with migration.
Why?
“Awesome, but convincing the managers is a nightmare ish task.”
“Do not see reason to remove working thing.”
Best practices
“I applaud all the hard work y'all have done around this. The one thing I'd really like to see is a maintainer recommendation for the "default" or "preferred" CRI to use, in addition to the suggestions of the various options. There's value in diversity for certain, but lacking an up front suggestion on the replacement can lead to confusion for organizations executing on the replacement.”
“Can you please provide best practice migration documentation.“
Links to vendor’s migration guides
“My k8s hosting provider does not support any other runtimes at this point” for GCP, AWS, Rancher
There seems to be issues with how Rancher supports runtimes, there were couple of comments about it.
Kubespray
“Thanks to Kubespray team, dockershim removal was quite easy. This community really rocks !“
“More documentation on how to do this for self hosted clusters would be very helpful, especially for Kubespray. At the moment I can only find info in GitHub issues”
Private registries support
“How to manage authentication with registry like AWS ECR for example ? Docker handled it with a plugin. Containerd does not seem to be ready to do it…”
Migration guides for kubeadm, k3s, kops. Eksctl
“We need documentation per cluster manager ( kubeadm, k3s, kops. Eksctl, .. so on)”
RHEL/CentOS7
“Not sure yet how to handle this in RHEL7/CentOS7 using official provided tools. Needs more research.”
Command line tools
“It's hard to understand whether the "docker" command-line tool will continue to work in containers running in such a cluster.“
Windows guidance
“More guidance for Windows container runtime migration will be helpful.”
“Windows kubelet dockershim migration path to containerd is unclear“
Kubenet future?
“use dockershim with kubenet cni, kubenet will be remove?|
Docs for buildkite
“The replacement cri-dockerd project is being neglected, which would otherwise have made this transition seamless for the user
Building in the single-node developer cluster was the reason for keeping docker, adding proper documentation for buildkitd is lacking from kubernetes”
CI/CD with non-docker environments or with Containerd when docker is installed on the box
Many comments are about CI/CD and how it will work without dockershim.
Lubomir I. Ivanov
>
>
> TL;DR; The number of people who don't feel prepared is over 40%. The most common reason for not migrating is a dependency on docker, in most cases those dependencies are from both - self-authored and third-party tools. 40% of respondents who do NOT feel ready for dockershim removal planned to adopt 1.24 in 2022. There are a list of tasks I compiled from free text comments. The need to re-educate personnel, change processes and tooling, without the direct business benefit (basically the cost of operation) is a big complaint that we also need to address as a community.
>
deprecation, ~55% of CR users were still using Docker:
https://github.com/kubernetes/kubernetes/pull/94624#issuecomment-689075442
based on older surveys this number was going down, but not drastically.
> Do you know that dockershim support will be removed in k8s 1.24?
>
> This is surprising that 17% weren't aware of the removal. This is one of the things discussed at sig contribex, and here https://github.com/kubernetes/community/issues/5344.
>
that talks about "What’s actually happening here is that Dockershim is
being removed from Kubelet as early as v1.23 release..."
https://kubernetes.io/blog/2020/12/02/dont-panic-kubernetes-and-docker/
1.24 is mentioned in the KEP, so the problem here might be that
updated release notes do not link to the KEP and 1.24 is not clear as
the timeline:
https://github.com/kubernetes/enhancements/blob/0e4d5df19d396511fe41ed0860b0ab9b96f46a2d/keps/sig-node/2221-remove-dockershim/README.md
at minimum we require *all users* to read release notes. k-dev,
twitter, blog posts are *more likely* to be missed.
> Do you feel you have enough information about your migration options and you prepared for the dockershim removal?
>
> The 40% respondents saying they do not feel prepared is a big number that calls for extra analysis.
>
and has potential for business downtime...which is a bit of a risk.
perhaps if https://github.com/Mirantis/cri-dockerd was maintained and
we had guides for in-place node migration using the external daemon it
would have made things easier for users that wish to continue using
docker for the time being and delay the migration to another CR.
>
> Comments analysis
>
> Quotes are posted as example of comments where they do not expose any PII. All comments will be shared with SIG Node chairs.
>
X in CR Y), but also saw a number of successful transitions to
containerd and cri-o.
overall, i think the honesty in the responses is this the comms and
reality check the k8s project needs when making big decisions.
the dockershim removal is one of the more (the most?) breaking changes in k8s.
i can see users getting stuck on version N-1 before N that removes it
even if we delay the removal by a few releases...
lubomir
--