Designate a label to indicate "node group"

[Max Williams]

Hi,

I created a Github issue for this but was told to post it here: https://github.com/kubernetes/kubernetes/issues/92137

I would like to have a node label designated for showing what "node group" a node belongs to. Currently GKE uses "cloud.google.com/gke-nodepool" and EKS uses "eks.amazonaws.com/nodegroup".

I know node groups don't really exist in the k8s API but they exist on every cloud platform in one form or another. We already have labels like "node.kubernetes.io/instance-type" which are similar in use.

Why is this needed? Having a designated common label will reduce administrative overhead in anything that relies on labelling, such as: any config/automation related to scheduling, Helm charts, anything prometheus related (Grafana dashboards, alerting etc), logging where labels are used and using kubectl "--label-columns" across clusters.

Maybe node.kubernetes.io/group?

Any feedback welcome.

Thanks,

Max

[Tim Hockin]

I don't have major objections to this, but I do wonder about the utility of it in practice.  What can we say about the value of this key?  Do we know that all providers are willing to live with a label value (very limited in content)?

Would we have in-project users of this or is it just for end-users?

What other gotchas are there?

--
You received this message because you are subscribed to the Google Groups "kubernetes-sig-architecture" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-arch...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-sig-architecture/5da4adaf-b0ec-4d6d-bd17-03db360c8d4cn%40googlegroups.com.

[Max Williams]

> I do wonder about the utility of it in practice.  What can we say about the value of this key?

The utility is in that it would be a recognised default way to identify a group of nodes. In my work with k8s across different teams, everyone is just having a stab at this label. I've seen "node_group", "worker-group", "worker-name" etc. This is annoying. It would be nice if "kubectl get nodes -L node.kubernetes.io/group" worked as commonly as "kubectl get pods -L app".

I can't really answer those other questions though 🙂

[Tim Hockin]

On Tue, Jan 26, 2021 at 6:56 AM 'Max Williams' via kubernetes-sig-architecture <kubernetes-si...@googlegroups.com> wrote:

> I do wonder about the utility of it in practice.  What can we say about the value of this key?

The utility is in that it would be a recognised default way to identify a group of nodes. In my work with k8s across different teams, everyone is just having a stab at this label. I've seen "node_group", "worker-group", "worker-name" etc. This is annoying. It would be nice if "kubectl get nodes -L node.kubernetes.io/group" worked as commonly as "kubectl get pods -L app".

Well, "app" is just a convention, not any sort of documented standard. :)

 

I can't really answer those other questions though 🙂

On Thursday, January 21, 2021 at 5:19:24 AM UTC+1 tho...@google.com wrote:

I don't have major objections to this, but I do wonder about the utility of it in practice.  What can we say about the value of this key?  Do we know that all providers are willing to live with a label value (very limited in content)?

Would we have in-project users of this or is it just for end-users?

What other gotchas are there?

On Fri, Jan 15, 2021, 4:11 AM 'Max Williams' via kubernetes-sig-architecture <kubernetes-si...@googlegroups.com> wrote:

Hi,

I created a Github issue for this but was told to post it here: https://github.com/kubernetes/kubernetes/issues/92137

I would like to have a node label designated for showing what "node group" a node belongs to. Currently GKE uses "cloud.google.com/gke-nodepool" and EKS uses "eks.amazonaws.com/nodegroup".

I know node groups don't really exist in the k8s API but they exist on every cloud platform in one form or another. We already have labels like "node.kubernetes.io/instance-type" which are similar in use.

Why is this needed? Having a designated common label will reduce administrative overhead in anything that relies on labelling, such as: any config/automation related to scheduling, Helm charts, anything prometheus related (Grafana dashboards, alerting etc), logging where labels are used and using kubectl "--label-columns" across clusters.

Maybe node.kubernetes.io/group?

Any feedback welcome.

Thanks,

Max

--
You received this message because you are subscribed to the Google Groups "kubernetes-sig-architecture" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-arch...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-sig-architecture/5da4adaf-b0ec-4d6d-bd17-03db360c8d4cn%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "kubernetes-sig-architecture" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-arch...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-sig-architecture/ede4e6d5-062c-402e-8c25-be372ed2097an%40googlegroups.com.

[Clayton Coleman]

Historically we've been pretty hesitant to endorse labels that imply topology.  node-role was a mixed bag and generated quite a lot of subtle breaks across different types of Kube deployments.  This label does have advantages over node-role in that it's an infrastructure characteristic ("the primary identifier of the infrastructure that provides these nodes") but not all such constructs would fit cleanly into the allowed label value space.  Do all infrastructure providers offer a human-readable primary identifier?  Would this be "human view only" (machine consumers are not allowed to assume anything about this)?

To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-sig-architecture/ede4e6d5-062c-402e-8c25-be372ed2097an%40googlegroups.com.

[Tim Hockin]

My assumption wrt label-value syntax was that this is proposed as a replacement for things that are ALREADY doing labels, just with ad hoc keys.  So the values are already syntactically valid.

The rest of your questions are appropriate.

To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-sig-architecture/CAH16ShLu2%2B9oahSNw7GEKRnqnbhRpbzJ4YEFE0rynXKNg%3D35%3DA%40mail.gmail.com.

[Max Williams]

> "app" is just a convention, not any sort of documented standard. :)

True but what about "app.kubernetes.io/name" then? This is specifically mentioned in the docs. Couldn't we just add a "node.kubernetes.io/group" in there?

> Do all infrastructure providers offer a human-readable primary identifier?

I'm not aware of a provider that does not have some string used for naming/grouping resources for cluster nodes. But there's so many new providers these days, I can't be sure. A provider could simply not add the label it doesn't fit their implementation though.

> Would this be "human view only" ?

I think so, yes. e.g. used by users for scheduling workloads onto specific groups of nodes.

> proposed as a replacement for things that are ALREADY doing labels, just with ad hoc keys

Yes, exactly.

Look, I'm just an end user, I don't really have knowledge or experience in maintaining the Kubernetes project itself :) I just wanted to ask if there was room for something better here as I have to deal with all these different implementations and it feels messy.

Can I safely use "node.kubernetes.io/group=XXXX" now and into the future?

[Tim Hockin]

This thread fell off the bottom of my inbox. To wrap it up I think
the answer is:

* There is no such normalized label today.

* We could add such a thing, but it would be, at best, an optional
convention (unreliable). If someone were to write a KEP (short)
describing this convention I could probably get behind it.

On Mon, Feb 8, 2021 at 2:52 AM 'Max Williams' via

> To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-sig-architecture/99459963-0b6a-46ad-a7de-a36fe4703b8an%40googlegroups.com.