Creating a new repository under k8s-sigs for new overcommit project

[Justyna Betkier (Sadło)]

Hi!

Today during the sig-autoscaling weekly meeting I brought a proposal to create a repository under k8s-sigs with sig-autoscaling sponsorship for a new project that would allow users to manage overcommit of their jobs. See high level proposal here: https://docs.google.com/document/d/1-_gSi-jDLsEvXr0YTiES5TUnPTygWiSZ8xYX6bAt1Zk/edit?usp=sharing

During the meeting there were no questions whether the project fits the sig-autoscaling scope (they may still come up as part of discussion here though), but rather whether the sig should sponsor greenfield, experimental projects or rather wait for them to have usage in order to get adopted by the sig.

The policy says that it is up to sig community to decide and there are known examples of other sigs doing this.

During the meeting we agreed that we will open the discussion on the mailing list and hence I am kicking it off here hoping that we can decide one way or another to unblock the prototype development.

Some context about the project state: there is already a concrete idea for the alpha/prototype stage (high level system behavior described in the proposal), we want to implement this as a prototype so that it can get usage and we can verify our assumptions. However we are planning full overhaul of the API before we go to beta as there are many open design decisions that are needed to propose a stable API. We want to start discussing use cases and ideas right away to incorporate them before we make further decisions for beta. We assume that for beta we would share the designs earlier for community discussions.

Best,

Justyna (jbtk)

[Jack Francis]

Hi Justyna,

I'm +1 on this problem statement being appropriate for SIG Autoscaling. It is possible that work in this area will uncover potential kubelet (SIG Node) work, but initially I think this is the right, and only SIG, that would represent this work.

What I'd like to see before we proceed with creating the new project structures (repos, CI, etc) is the following:

- Who is committing to maintaining the project? (i.e., OWNERS file data)

- What is the TTL on retiring the project if it doesn't graduate past alpha?

I think we would also want to ensure that beta (and potentially v1) graduation includes a strong commitment to maintain long-term, but we don't have to figure that out now. FYI.

Generally excited to host a greenfield effort in the SIG to solicit contributions from the public, and to potentially eliminate duplicate silo efforts addressing the same problem.

Jack

[Justyna Betkier]

- Who is committing to maintaining the project? (i.e., OWNERS file data)

I would propose having an OWNERS file with Krzysiek (@krzysied), Brandon (@bwsalmon) and myself (@jbtk) for a start. If we would like to also add someone more experienced in the community I also talked with Daniel (@x13n) and he agreed to be also included in this repository.

Krzysiek has been a member of k8s community for quite some time now, he used to work on cluster scalability and in recent years he was focused mostly on internal GKE cluster autoscaler. Brandon came up with the idea of overcommit. He is active in a space of efficiency, mostly between scheduling and autoscaling. I was engaged in the CapacityBuffers proposal and getting it to beta, I am also active in discussions about the future of the scheduling and autoscaling in the SDA effort.

- What is the TTL on retiring the project if it doesn't graduate past alpha?

After discussing offline with Jack I was suggested to propose 2 years - while we hope it will be faster to reach beta depending on the user needs and community discussions we may end up in KEP discussions with node and waiting for some core k8s features to unblock us. We still have some unknowns and things that we do not fully understand around the memory accounting on the node and we expect that as push utilization on the not higher we may end up breaking some assumptions that used to be true before and uncovering rough edges (or worse :P) 

Best,

Justyna

--
You received this message because you are subscribed to the Google Groups "Autoscaling Kubernetes" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-auto...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/kubernetes-sig-autoscaling/77ca14f8-0bc2-4a05-8ff0-76669164568an%40googlegroups.com.