What should happen when a feature gate goes to GA?

[Jordan Liggitt]

We've had a few instances of feature gates that graduated to GA, and a few more in the works.

There are some questions about what should happen with a gate and the conditional nature of the code paths it controls when a feature gate graduates to GA:

1. Should the feature gate remain a valid option to set (shows up in help, `--feature-gates=Foo=true` is allowed, etc)?

2. Should setting it to false be allowed, or should GA gates only be allowed to be true?

3. Should setting it to false still disable relevant code paths?

[Davanum Srinivas]

One data point.

CoreDNS was marked GA in 1.11 [1]. We still have code paths active so it can be set to false [2]. Because of criteria from KEP [3] 

[1] https://github.com/kubernetes/features/issues/427

[2] https://cs.k8s.io/?q=features%5C.CoreDNS&i=nope&files=&repos=kubernetes/kubernetes

[3] https://github.com/kubernetes/features/issues/566#issuecomment-410735394

Thanks,

Dims

--
You received this message because you are subscribed to the Google Groups "kubernetes-sig-architecture" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-arch...@googlegroups.com.
To post to this group, send email to kubernetes-si...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-sig-architecture/CALbx6sYszA-4rds-x7FmQkLruHjjzBOppe%2B23LFd6KvA-X8ugw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--

Davanum Srinivas :: https://twitter.com/dims

[Daniel Smith]

I think our eventual goal will need to be to reduce the space of operating configurations-- we're definitely not going to test everything with every combination of such flags.

I think GA features should in general be on by default; the flag should permit mentioning them (to avoid breaking configs) but ignore the setting & warn if it is set. Once a feature flag name is used for something it should never be reused.

I think there may be some behaviors that users legitimately might want on or off, but we seem to manage such things fine with other mechanisms.

[Tim Hockin]

Agree with Daniel. For things users legit want on/off the gate (and
development) should cover that mechanism, not BE that mechanism.
On Fri, Aug 24, 2018 at 11:10 AM 'Daniel Smith' via
kubernetes-sig-architecture

> To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-sig-architecture/CAB_J3bbAFuC0dHeEojCRGs2GMuFiU4JW6hpftkjyjF7rWB7YNQ%40mail.gmail.com.

[Brian Grant]

This should be covered by our deprecation policy: https://kubernetes.io/docs/reference/using-api/deprecation-policy/

Rule #5b: CLI elements of admin-facing components (e.g. kubelet) must function after their announced deprecation for no less than:

...

Rule #6: Deprecated CLI elements must emit warnings (optionally disable) when used.

On Fri, Aug 24, 2018 at 11:10 AM 'Daniel Smith' via kubernetes-sig-architecture <kubernetes-si...@googlegroups.com> wrote:

I think our eventual goal will need to be to reduce the space of operating configurations-- we're definitely not going to test everything with every combination of such flags.

I think GA features should in general be on by default; the flag should permit mentioning them (to avoid breaking configs)

Yes, we should not break configs.

 

but ignore the setting & warn if it is set. Once a feature flag name is used for something it should never be reused.

There are valid use cases for turning off APIs, so we may need to qualify that.

 

I think there may be some behaviors that users legitimately might want on or off, but we seem to manage such things fine with other mechanisms.

On Fri, Aug 24, 2018 at 10:52 AM Jordan Liggitt <jlig...@redhat.com> wrote:

We've had a few instances of feature gates that graduated to GA, and a few more in the works.

There are some questions about what should happen with a gate and the conditional nature of the code paths it controls when a feature gate graduates to GA:

1. Should the feature gate remain a valid option to set (shows up in help, `--feature-gates=Foo=true` is allowed, etc)?

2. Should setting it to false be allowed, or should GA gates only be allowed to be true?

3. Should setting it to false still disable relevant code paths?

--
You received this message because you are subscribed to the Google Groups "kubernetes-sig-architecture" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-arch...@googlegroups.com.
To post to this group, send email to kubernetes-si...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-sig-architecture/CALbx6sYszA-4rds-x7FmQkLruHjjzBOppe%2B23LFd6KvA-X8ugw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "kubernetes-sig-architecture" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-arch...@googlegroups.com.
To post to this group, send email to kubernetes-si...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-sig-architecture/CAB_J3bbAFuC0dHeEojCRGs2GMuFiU4JW6hpftkjyjF7rWB7YNQ%40mail.gmail.com.

[Daniel Smith]

On Fri, Aug 24, 2018 at 11:37 AM Brian Grant <brian...@google.com> wrote:

This should be covered by our deprecation policy: https://kubernetes.io/docs/reference/using-api/deprecation-policy/

Rule #5b: CLI elements of admin-facing components (e.g. kubelet) must function after their announced deprecation for no less than:

...

Rule #6: Deprecated CLI elements must emit warnings (optionally disable) when used.

On Fri, Aug 24, 2018 at 11:10 AM 'Daniel Smith' via kubernetes-sig-architecture <kubernetes-si...@googlegroups.com> wrote:

I think our eventual goal will need to be to reduce the space of operating configurations-- we're definitely not going to test everything with every combination of such flags.

I think GA features should in general be on by default; the flag should permit mentioning them (to avoid breaking configs)

Yes, we should not break configs.

 

but ignore the setting & warn if it is set. Once a feature flag name is used for something it should never be reused.

There are valid use cases for turning off APIs, so we may need to qualify that.

We already have everyone's favorite flag --runtime-config for that; also CRDs & APIServices are super easy to turn off.

Aside: I think the worst thing about the feature gates is that they're built wrong, they're plumbed separately into every binary in the cluster.

[Jordan Liggitt]

I think our eventual goal will need to be to reduce the space of operating configurations-- we're definitely not going to test everything with every combination of such flags.

I strongly agree with this

I think GA features should in general be on by default;

Agree

the flag should permit mentioning them (to avoid breaking configs) but ignore the setting & warn if it is set.

That means the flag no longer functions, which doesn't mesh with the stated deprecation policy. Marking and announcing the feature gate as deprecated when it goes from beta to GA, then removing it and all checks of it a release later would cover the "3 months or 1 release" requirement for "CLI elements of admin-facing components".

I think there may be some behaviors that users legitimately might want on or off, but we seem to manage such things fine with other mechanisms.

Makes sense. This is where we ended up on things like kubelet client cert rotation (actual config option in addition to the on-by-default-and-eventually-removed feature gate), which depends on the cluster deployer having set up approval/signing processes.

[Tim Allclair]

This is discussed in https://github.com/kubernetes/kubernetes/issues/46404

On Tue, Aug 28, 2018 at 7:00 AM Jordan Liggitt <jlig...@redhat.com> wrote:

I think our eventual goal will need to be to reduce the space of operating configurations-- we're definitely not going to test everything with every combination of such flags.

I strongly agree with this

I think GA features should in general be on by default;

Agree

the flag should permit mentioning them (to avoid breaking configs) but ignore the setting & warn if it is set.

That means the flag no longer functions, which doesn't mesh with the stated deprecation policy. Marking and announcing the feature gate as deprecated when it goes from beta to GA, then removing it and all checks of it a release later would cover the "3 months or 1 release" requirement for "CLI elements of admin-facing components".

I would argue this process starts as soon as the feature goes to beta. IMO feature gates should be considered a special case. For example, what if the feature is actually deprecating some other feature (see advanced auditing).

I agree that setting a GA feature gate should not be an error (I.e. configs referencing it shouldn't break), but I think making it a no-op should be acceptable.

 

I think there may be some behaviors that users legitimately might want on or off, but we seem to manage such things fine with other mechanisms.

Makes sense. This is where we ended up on things like kubelet client cert rotation (actual config option in addition to the on-by-default-and-eventually-removed feature gate), which depends on the cluster deployer having set up approval/signing processes.

--
You received this message because you are subscribed to the Google Groups "kubernetes-sig-architecture" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-arch...@googlegroups.com.
To post to this group, send email to kubernetes-si...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-sig-architecture/CALbx6sZ8e5KsYDsTf0%2BCmxDp5pkuC1SVHSysg_UcPkBz5VUKxA%40mail.gmail.com.

[Jordan Liggitt]

On Thu, Aug 30, 2018, at 7:30 PM, 'Tim Allclair' via kubernetes-sig-architecture wrote:

I would argue this process starts as soon as the feature goes to beta.

That makes sense for people who were previously specifying the gate to enable an alpha feature... after all, it's enabled by default now, and they have a release to stop explicitly turning it on.

It makes less sense for people who explicitly specify it in order to disable the beta feature (because it doesn't meet their requirements, or hasn't been tested in their environment or at their scale, or exposes function that doesn't have corresponding security controls in place, or because they're curmudgeons who don't like shiny new features).

For those people, ideally, we'd announce deprecation of the gate a release before it moved to GA, then remove it at GA (the GA threshold implies the feature has taken into account various environments, and reached an acceptable level of testing, security controls, use, etc).

However, not being able to tell the future, the next best approach seems to me to be to deprecate the gate when it goes GA, honor it for one more release, then remove it.

[Davanum Srinivas]

Team,

Silvia (@Sil on slack) has a PR up for review. Let's please move the discussion there and hopefully get it out for 1.12 docs.

https://github.com/kubernetes/website/pull/10294/

Thanks,

Dims 

--

You received this message because you are subscribed to the Google Groups "kubernetes-sig-architecture" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-arch...@googlegroups.com.
To post to this group, send email to kubernetes-si...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-sig-architecture/1535682716.2801401.1492059632.5F12A699%40webmail.messagingengine.com.

For more options, visit https://groups.google.com/d/optout.

[Davanum Srinivas]

Dear Arch Folks,

Can we please get +1(s) on this? So we can ship it with 1.12 docs.

https://github.com/kubernetes/website/pull/10294

Thanks,

Dims