Guidance for *.k8s.io APIs

[Jordan Liggitt]

There have been recent discussions about the use of *.k8s.io APIs in combination with CRD-based development and out-of-tree SIG subprojects. This is on the agenda for the next sig-arch meeting, but I wanted to start the conversation here.

Some of the questions:

• Can SIG subprojects use *.k8s.io (including *.sigs.k8s.io) APIs with CRDs? (I would expect so, though *.sigs.k8s.io is not likely to be meaningful to API consumers)
• If so, do those APIs require API review? (yes)
• What is the naming guidance (API group suffix, version, etc) for SIG subproject REST or config APIs that intend to be broadly consumed and treated as stable API targets (e.g. cluster API)?
• What is the naming guidance (API group suffix, version, etc) for SIG subproject REST or config APIs that are "internal"-facing and don't intend to offer compatibility/support guarantees (e.g. config files for internal tools)?
  
• How do we prevent accidental use of *.k8s.io-namespaced APIs by things outside kubernetes and kubernetes sigs? (see open KEP for this)
  

My primary goals for Kubernetes APIs are (roughly in descending order of importance):

Clear user expectations

Most users don't know/care about SIGs. It should be clear to a user of a *.k8s.io API what the support/deprecation policy is. I think it's reasonable for most users to expect the existing project-wide API version and deprecation policies[1][2] to apply.

Compatibility (for a particular custom resource)

Development of *.k8s.io APIs (built-in or CRD-based) should support our compatibility and deprecation goals. For CRD-based development in particular, the tooling that generates them (things like kubebuilder) does not currently have guardrails that tell you if you're making incompatible changes to a CRD's schema over its lifetime.

Quick iteration during development

One of the benefits of custom resources over built-in resources is the ability to rapidly iterate. How can we structure the review process to support this rapid iteration, while drawing a clear line between supported/unsupported APIs? A tension is that historically, once an API (even an alpha, unsupported one) has *any* consumers, resistance to change it increases dramatically.

Consistency

API conventions[4] have developed over that lifetime of the project. Some of those conventions are built into the CRD server so that all custom resources conform to them, but many still depend on CRD schema authors to be aware of the conventions and shape APIs consistently.

References:

[1] Kubernetes API version definitions

[2] Kubernetes deprecation policy

[3] API review policy

[4] API conventions

[5] KEP proposing CRDs for *.k8s.io APIs point to where they were approved

[Tim Hockin]

On Tue, Jul 2, 2019 at 12:28 PM 'Jordan Liggitt' via
kubernetes-api-reviewers <kubernetes-a...@googlegroups.com>
wrote:

>
> There have been recent discussions about the use of *.k8s.io APIs in combination with CRD-based development and out-of-tree SIG subprojects. This is on the agenda for the next sig-arch meeting, but I wanted to start the conversation here.
>
> Some of the questions:
>
> Can SIG subprojects use *.k8s.io (including *.sigs.k8s.io) APIs with CRDs? (I would expect so, though *.sigs.k8s.io is not likely to be meaningful to API consumers)
> If so, do those APIs require API review? (yes)
> What is the naming guidance (API group suffix, version, etc) for SIG subproject REST or config APIs that intend to be broadly consumed and treated as stable API targets (e.g. cluster API)?
> What is the naming guidance (API group suffix, version, etc) for SIG subproject REST or config APIs that are "internal"-facing and don't intend to offer compatibility/support guarantees (e.g. config files for internal tools)?

I think this guidance should extend to annotations and labels, too.
We have a proliferation of them, with little to no curation, and no
canonical registry. Historically cloud-provider code has used
*.kubernetes.io annotations because "well, it's in the kubernetes tree
and we don't own mega-provider.com".

> How do we prevent accidental use of *.k8s.io-namespaced APIs by things outside kubernetes and kubernetes sigs? (see open KEP for this)
>
> My primary goals for Kubernetes APIs are (roughly in descending order of importance):
>
> Clear user expectations
> Most users don't know/care about SIGs. It should be clear to a user of a *.k8s.io API what the support/deprecation policy is. I think it's reasonable for most users to expect the existing project-wide API version and deprecation policies[1][2] to apply.
>
> Compatibility (for a particular custom resource)
> Development of *.k8s.io APIs (built-in or CRD-based) should support our compatibility and deprecation goals. For CRD-based development in particular, the tooling that generates them (things like kubebuilder) does not currently have guardrails that tell you if you're making incompatible changes to a CRD's schema over its lifetime.
>
> Quick iteration during development
> One of the benefits of custom resources over built-in resources is the ability to rapidly iterate. How can we structure the review process to support this rapid iteration, while drawing a clear line between supported/unsupported APIs? A tension is that historically, once an API (even an alpha, unsupported one) has *any* consumers, resistance to change it increases dramatically.
>
> Consistency
> API conventions[4] have developed over that lifetime of the project. Some of those conventions are built into the CRD server so that all custom resources conform to them, but many still depend on CRD schema authors to be aware of the conventions and shape APIs consistently.
>
>
> References:
> [1] Kubernetes API version definitions
> [2] Kubernetes deprecation policy
> [3] API review policy
> [4] API conventions
> [5] KEP proposing CRDs for *.k8s.io APIs point to where they were approved
>

> --
> You received this message because you are subscribed to the Google Groups "kubernetes-api-reviewers" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-api-rev...@googlegroups.com.
> To post to this group, send email to kubernetes-a...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-api-reviewers/CAMBP-pKefC52VQnM9wXS75RaMCNm7LpE-FkAJ70z3kfk-ZuThQ%40mail.gmail.com.
> For more options, visit https://groups.google.com/d/optout.

[Tim St. Clair]

Much like we created org structure to support the expanded ecosystem of kubernetes, we need to help provide guidance and policy that helps empower and facilitate the tools in that structure. 

There is an order of magnitude greater extensions today then there are core api's and helping to set best practices will facilitate the community.  Within sig-cluster-lifecycle we have a number of sub-projects who are creating CRD based tools, namely "AddOns, and Cluster API".  We want to follow best practices, but we also do not want to be encumbered by bureaucracy or to be gated by a few overbooked individuals, hence the reason they are sub-projects and not core.

What we have been asking for are recommendations for the ecosystem (docs) on naming so expectations are clear to the consumers of those apis.

--
You received this message because you are subscribed to the Google Groups "kubernetes-sig-architecture" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-arch...@googlegroups.com.
To post to this group, send email to kubernetes-si...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-sig-architecture/CAMBP-pKefC52VQnM9wXS75RaMCNm7LpE-FkAJ70z3kfk-ZuThQ%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Cheers,
Timothy St. Clair

“Do all the good you can. By all the means you can. In all the ways you can. In all the places you can. At all the times you can. To all the people you can. As long as ever you can.” 

[Joe Beda]

This is a good discussion for us to be having.

 

I’ve put some of my thoughts into a github issue that we can discuss.  Happy to turn this into a PR against the API review process document.

 

https://github.com/kubernetes/community/issues/3864

 

Joe

--
You received this message because you are subscribed to the Google Groups "steering" group.
To unsubscribe from this group and stop receiving emails from it, send an email to steering+u...@kubernetes.io.
To view this discussion on the web visit https://groups.google.com/a/kubernetes.io/d/msgid/steering/CALM%2Bqp9caOrN32624aFV_mJ%3DU7FTzHE4SvUd6hgeWHaD%2BQ2zxA%40mail.gmail.com.

[Matt Farina]

Two quick things....

1) A doc to come up with consumer/community facing direction has been started at https://docs.google.com/document/d/1nWLC_JISeuAAaMF0eRWftbGETef7u-52jzAK4PjedU0/edit#

2) For those interested in an update, in SIG Arch there were no objections and we are going to move ahead with the proposal. It is in a merge and iterate stage. If you still have an objection or issues share on the relevant PRs.