Kubeconfig File Locking

[Ross Peoples]

Hello all,

During the latest SIG-CLI meeting, we discussed the following PR:

https://github.com/kubernetes/kubernetes/pull/107354

One of the issues brought up during discussions was the introduction of a new dependency in client-go. On the call, we determined that we could easily reimplement that functionality without the additional dependencies.

However, the question was also asked if we wanted to remove file locking entirely, since the general guidance is to not make concurrent changes to kubeconfig. The goal here is to get some feedback from the community on whether we should continue to support file locking via a more robust and atomic mechanism, or should we phase out file locking kubeconfig entirely.

Please let me know your thoughts. Thanks,

Ross

[Davanum Srinivas]

+1 to phase out file locking!

--
You received this message because you are subscribed to the Google Groups "dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dev+uns...@kubernetes.io.
To view this discussion on the web visit https://groups.google.com/a/kubernetes.io/d/msgid/dev/CAPvcVKZdDxqJAPBdmjm9qN%2BUWckHPTmsuUiJEKFUmo4YcdkZyg%40mail.gmail.com.

--

Davanum Srinivas :: https://twitter.com/dims

[Taahir Ahmed]

However, the question was also asked if we wanted to remove file locking entirely, since the general guidance is to not make concurrent changes to kubeconfig.

How are users supposed to prevent themselves from concurrently modifying kubeconfig, in the general case?  Many tools will modify it as a side-effect. 

On Wed, Feb 9, 2022 at 2:55 PM Jason DeTiberus <jdeti...@packet.com> wrote:

To echo what Ben said, there is a lot of tooling out there that interacts with the '/etc/kubernetes/admin.kubeconfig', from home grown terraform/ansible scripts to various other projects and products that do k8s cluster lifecycle management.

I think prematurely removing support for the file locking without having some type of a deprecation period and alternative for users that are relying on it for avoiding external tooling (and local users) from stepping on each other.

--

Jason DeTiberus

On Wednesday, February 9, 2022 at 5:51:09 PM UTC-5 Benjamin Elder wrote:

FWIW, I think it would be easier to phase out locking if we had better support for multiple kubeconfig files and split out the current context (https://github.com/kubernetes/kubectl/issues/569#issuecomment-1033123302), but I don't think there's any progress on this. 

Right now, I'm not sure how software like sigs.k8s.io/kind should prevent merging bugs without locking. 

We could just write to many distinct files ourselves, but users expect clusters to "just work" and be configured without setting `KUBECONFIG=foo:bar:baz` and a number of deployment tools are writing to kubeconfig with client-go today.

To view this discussion on the web visit https://groups.google.com/a/kubernetes.io/d/msgid/dev/CANw6fcH_OVSP_OnA%3DfoWmXhb7_sbJ3Gz_YbpfCrns70rCLK-vw%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dev+uns...@kubernetes.io.

To view this discussion on the web visit https://groups.google.com/a/kubernetes.io/d/msgid/dev/535faebb-567c-4ecd-b838-57d7032e457fn%40kubernetes.io.

[Benjamin Elder]

FWIW, I think it would be easier to phase out locking if we had better support for multiple kubeconfig files and split out the current context (https://github.com/kubernetes/kubectl/issues/569#issuecomment-1033123302), but I don't think there's any progress on this. 

Right now, I'm not sure how software like sigs.k8s.io/kind should prevent merging bugs without locking. 

We could just write to many distinct files ourselves, but users expect clusters to "just work" and be configured without setting `KUBECONFIG=foo:bar:baz` and a number of deployment tools are writing to kubeconfig with client-go today.

On Wed, Feb 9, 2022 at 1:59 PM Davanum Srinivas <dav...@gmail.com> wrote:

To view this discussion on the web visit https://groups.google.com/a/kubernetes.io/d/msgid/dev/CANw6fcH_OVSP_OnA%3DfoWmXhb7_sbJ3Gz_YbpfCrns70rCLK-vw%40mail.gmail.com.

[Jason DeTiberus]

To echo what Ben said, there is a lot of tooling out there that interacts with the '/etc/kubernetes/admin.kubeconfig', from home grown terraform/ansible scripts to various other projects and products that do k8s cluster lifecycle management.

I think prematurely removing support for the file locking without having some type of a deprecation period and alternative for users that are relying on it for avoiding external tooling (and local users) from stepping on each other.

--

Jason DeTiberus

On Wednesday, February 9, 2022 at 5:51:09 PM UTC-5 Benjamin Elder wrote:

[Ken Sipe]

It is great to get community feedback!  Would it also make sense to gather data regarding projects that leverage client-go and/or kube config file?   I would expect there are a significant number of projects that have expectations on kubeconfig.  The lock isn’t just about in-project protection.  It also is cross tool collaboration feature.   Perhaps we could get feedback from CNCF projects (which may not find this email thread).

I am for a platform specific solution.  However it is useful to note that there is a Golang design proposal to expose this functional from Go (currently buried under /internal).  Details were posted on the GH issue noted at the start of this email thread.

Thanks for driving this!

Ken