Deadlines are horrible

[Tim Hockin]

My dearest sig-release heroes and heroines, can I bend your ears for
just a second?

I know the deadlines are there to try to get better releases. In
practice, here's what I am seeing:

For a month there's a steady trickle of KEPs. We go back and forth on
them, they get priority whenever people can get to them.

Then 3-4 days before the KEP deadline, I start getting pings. Email,
github, slack, hangouts, twitter. I do my best to get to them but I
have other work to do and I have a life and family, so I am not really
responding all day Saturday and Sunday.

Then, the day before the deadline, my pocket is buzzing NON-STOP. I
am furiously reviewing KEPs, still finding problems and having
questions. Now I feel bad -- if they miss the boat because of me that
will be awful. So I am in a virtual panic all day long. I am trying
to make it down the queue, but some of these are big, and some require
careful reading.

I have reviewed some of these KEPs 3 times _today_. It's not
reasonable to assume anyone can do this. The faster I go, the more I
miss.

How do we break the slide-it-under-the-door-before-8am thing going on here?

The fact that I am raising this many concerns this late in the game
says to me that something is wrong. For every 1 KEP I review there
are 3-4 other I don't. We have a great pool of reviewers, but this...
this is exhausting. I have done almost nothing else all day and for
the past few days.

Tim

[Davanum Srinivas]

This sucks big time!!! :(

--
You received this message because you are subscribed to the Google Groups "kubernetes-sig-release" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-re...@googlegroups.com.
To post to this group, send email to kubernetes-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-sig-release/CAO_RewZkYLfBpiUfXSB%3Do1mTh0zpRqA3Nk1fggx99JB2B%2BJDhg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--

Davanum Srinivas :: https://twitter.com/dims

[bbu...@microsoft.com]

Tim,

totally sorry that I was one of the people bugging you :(.

I think this points to the fact that KEP reviews in the SIGs aren't scaling. We need to broaden/improve the process so that we both track how KEPs are processing through SIGs and get a bigger pool so that they move faster.

In the case of the KEP that I'm interested in, it's been open and being semi-reviewed for some time now (months) but w/o the deadline there's no urgency on the part of the SIGs to review it, and thus little velocity.

So we need some way to ensure velocity, without forcing emergencies with deadlines...

Anyway, apologies again for being one of the people who bug you...

--brendan

To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-release+unsub...@googlegroups.com.

To post to this group, send email to kubernetes-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-sig-release/CAO_RewZkYLfBpiUfXSB%3Do1mTh0zpRqA3Nk1fggx99JB2B%2BJDhg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Stephen Augustus]

(+SIG PM, SIG Arch, SIG Chairs/Leads)

[Putting my SIG PM hat on, as KEPs are a PM concern]

First things first, let me just say, "Dang, that sucks.".

No one should have to worry about working on the weekends or taking time away from their family and I'm sorry you're having to deal with that right now.

As Brendan mentioned, this is a scaling problem. Multiple vectors/personas to consider though.

A concrete first step for all SIG Chairs/Technical Leads to take is to update your KEP subdirectory OWNERS with additional reviewers/approvers. If you have people you trust to review designs for your SIG, I would definitely suggest adding them in k/enhancements, as SIG Chairs/Technical Leads are currently in the critical path for all KEP approvals. I mentioned this on the Community call last week and a few other places, but maybe haven't written it down somewhere.

Now some of the other vectors...

Enhancements Lead

Enhancements tracking is a process that leaves a lot to be desired. 

It's a spreadsheet that I (and several others) want to kill with fire.

That spreadsheet is created by a single human (Enhancements Lead) [and sometimes some shadows] whose job is to read through 142 enhancement tracking issues and make a call on whether or not that work should be tracked for the cycle. For someone in this role newer to the project/process, they may not have the context/time/technical understanding to easy glean what the status for one SIG, much less 31(?).

Release Team

The Enhancements Lead takes each of these issues and lovingly files them in the Enhancements Tracking spreadsheet, which becomes the source of truth for the Release Team and anyone else who looks at it for the cycle. That means they're also tasked with watching all of the updates to k/enhancements and constantly reconciling that information back to the spreadsheet. Ihor did this for multiple cycles, I did it for two, and Kendrick is in his second cycle doing it now.

This is one of the reasons that it's so crucial for us to have an accurate description in enhancement issues. Anything that's wrong there cascades as bad information to the Release Team.

KEP Approvers

As mentioned above, KEP reviewers/approvers are currently only SIG Chairs/Technical Leads, which means in addition to all of the stuff you're already reviewing, you become the bottleneck for merging KEPs.

PM

We're unable to synthesize any thoughts about what a roadmap is, themes for the release, etc. because the information waterfalls in at the buzzer, which suggests that we need to frontload some sort of process before all of these personas are scrambling.

Okay, so my thoughts on your thoughts...

KEPs are a steel thread. They attempt to capture all of the intent of an enhancement in a single place.

A user, consumer, developer, provider of anything Kubernetes or Kubernetes-adjacent should be able to read one and understand exactly what an enhancement does, where the docs are, what the graduation criteria is, how it's tested, how it's implemented, and what we could've done instead without much prior knowledge of it, or the need to dig in various repos to formulate context. It should be the essence/intent of the code we're about to write... like writing tests before implementation.

Imagine a passerby being able to host the CNCF Release Webinar and just read off KEPs that were so beautifully written that there was no need for questions. Pipe dream I know, but that's the quality I have in mind.

KEP PRs should be a continuous stream, unbounded by release cycles.

Merge early, iterate often.

There are currently 61 PRs open in k/enhancements, which should never happen.

I can almost guarantee that a vast majority of those are KEPs that were marked as `provisional`, but are being held to the standard of `implementable`, and are getting bikeshed into inactivity.

Let's get out of the habit of doing this. 

If the idea is good and you think it's something your SIG might go forward with, get the summary, motivation, goals/non-goals merged as provisional and then make a plan of attack with the KEP author to move it to implementable.

Also keep in mind that there are other KEP statuses, which may better suit the ideas need: provisional|implementable|implemented|deferred|rejected|withdrawn|replaced

As Brendan mentioned, there's a bit of a stick here. 

Not providing the appropriate information by the deadline requires that SIGs pass another hurdle i.e., filing an exception. 

I view that as a way to protect the Release Team's time, but in future, maybe we can use it to gain visibility in slower moving SIGs or areas of code.

While it's neither a goal or requirement for SIG PM to dictate meeting structure or other SIG's events, I do think there's value in considering SIG planning events and grooming KEPs as one would a "regular" issue/PR.

TSC and SIG Cluster Lifecycle set an excellent example here: https://github.com/kubernetes/community/blob/master/sig-cluster-lifecycle/grooming.md

I see several of their KEPs essentially written as a group activity and the approval and merge is simply a matter of ceremony/committing a KEP to git history.

If a SIG planning event happened around code freeze of the previous cycle, a SIG could define exactly what was going into the next release. 

KEP review cycles would only be spent on KEPs that the SIG had committed to for that milestone/epic/sprint.

I think a KEP author could respect/accept a SIG Chair/TL saying, "Hey, sorry, this is not something we have bandwidth for this cycle. It's in the hopper though, and if you give me this KEP with foo, bar, and baz in it by X date, then it's got a good chance of getting reviewed/approved for 1.n+1."

SIG Chairs/TLs should feel comfortable and enabled to push back like that because it'd be something built into their process and documented.

Having that kind of conversation early then trickles down and gives the Release Team breathing room to capture everyone's intent well ahead of schedule and focus instead on automating lots of frivolous things in the process away.

Anywho, I'm rambling now, but I hope it was helpful, and I hope you know we're here to make the load lighter.

Please continue to provide feedback like this because it helps everyone!

-- Stephen

To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-re...@googlegroups.com.

To post to this group, send email to kubernetes-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-sig-release/CAO_RewZkYLfBpiUfXSB%3Do1mTh0zpRqA3Nk1fggx99JB2B%2BJDhg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--

Davanum Srinivas :: https://twitter.com/dims

--

You received this message because you are subscribed to the Google Groups "kubernetes-sig-release" group.

To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-re...@googlegroups.com.

To post to this group, send email to kubernetes-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-sig-release/08183e9c-7496-4189-a4fc-24b0fc0e1746%40googlegroups.com.

[Andrew Kim]

First of all, I want to say thank you (and sorry) to folks who have been diligently reviewing/approving KEPs!

From my experience being a SIG Chair/TL, I struggle a lot with knowing when I am in a position to approve a KEP. If I’m a SIG TL but not an API/code approver, can I still approve KEPs that will eventually require approval from the appropriate API approvers? I’m not sure..... as much as I want the context/expertise, the reality is that it's safer to lean on the experience/expertise of a veteran like Tim Hockin. The best I can do is shadow the existing API approvers for now, but until I become an API approver myself, I will likely defer to the current API approvers for KEPs.

Maybe one of the problems is not needing more KEP approvers, but more API approvers. In which case it seems like we are actively working on that already.

Hope my perspective helps!

- Andrew Sy Kim

You received this message because you are subscribed to the Google Groups "kubernetes-sig-architecture" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-arch...@googlegroups.com.
To post to this group, send email to kubernetes-si...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-sig-architecture/CAFQm5yR-xv3gJ%3DBWyJBUSu3NLJqqprz8v-erkn43S4%3DG9-wTwQ%40mail.gmail.com.

[Stephen Augustus]

I think they're two similar but separate problems, funneling into a common set of bandwidth-constrained reviewers.

I have limited (non-existent) experience with k8s API review, but I can say with some confidence that they're not the reason for 61 open KEPs.

Completely agree that I too (at this stage, at least) would shy away anything that resembled API review because there are others with better context and history in the project.

[Brendan Burns]

I think as a a SIG reviewer you should feel comfortable approving a KEP with an API change.

I think that any KEP with an API change should also be going through a separate API review (SIG-Arch), so approval of the KEP from a SIG like networking doesn't necessarily imply API approval for that KEP.

--brendan

---

From: Stephen Augustus <Ste...@agst.us>
Sent: Wednesday, May 1, 2019 4:14 PM
To: Andrew Kim
Cc: Tim Hockin; kubernetes-sig-leads; Brendan Burns; Davanum Srinivas; kubernet...@googlegroups.com; kubernetes-sig-architecture; kubernetes-sig-release; Kubernetes Release Team
Subject: Re: Deadlines are horrible

 

[Derek Carr]

I think the model employed by SIG Cluster Lifecycle to have a SIG meeting and a separate office hour meeting is a good model when there is a consistent set of attendees.  It does help for things like issue triage across the group and depending on everyone else to help inform everyone about things that get lost in GitHub notifications.  I think it’s a model we could follow in SIG Node.

A struggle I have observed in SIG Node is enhancement ideas that come from new (which is great!) or less frequent contributors (also great) that may span a few SIGs.  I still see a lot of confusion about what is required in a KEP, when a KEP is needed, and the appropriate level of depth (especially around testing and graduation criteria).  If it’s a new contributor, I want to do the best I can to shepherd the work, but I am sensing a frustration point where folks are still saying “I’m new to this KEP thing, I still don’t really know what it is I have to do next, why is this so hard?  Just look at the PR that implemented it!”.  I’m not sure what we can do to make this better, but I can empathize with the comments in this thread where it can be a large quantity to review and reason about and a frustration point for those looking to just make a change.  In SIG Node we try to map an owner with a reviewer when discussing plans, but then new late items arise outside of that process often from new folks wanting to help out.

I like KEPs, it’s actually been really helpful to maintainers, and I really want to see them used more consistently across all sub-projects, but on-boarding folks to the KEP process is still difficult.  Merging KEPs in a provisional state is good, but unless the questions unanswered are clear in the merged doc, it can be a difficult context switch when reviewing follow-on PRs.

I have wondered if we can capture some type of change budget within a SIG or afford more time from end-users to comment on proposed changes somehow.  I sometimes worry that in aggregate across all SIGs and platforms, the amount of change and it’s impact to end-users is not always well-accounted.

Either way, I would love to reach the state that Stephen aspired to for all of our KEPs while keeping the process fun and engaging for all contributors.

- Derek

To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-sig-architecture/CABc050F17GTYcN%3DOvkTVAoytBqVcA%3Dpzzm3hsK8dKtLM9iWhSg%40mail.gmail.com.

[Niko Penteridis]

Process-wise, I think the concept of Slush -> Freeze would work well on the Enhancements cycle.

Enhancements Freeze has been scheduled at the 4th week of the release cycle in the past X (many) releases. Perhaps that is a tad too fast, and while it's true that the rest of the time is free to prepare for the next cycle, I think that people are generally not mentally prepared for a hard-stop-freeze at that date, both submitters and reviewers.

What usually happens:

- People forget the deadline date

- Timezones are (very) tricky

- Warning given is sometimes only 2-3 days before freeze, and might be simply lost in GH notifications

- Due to the above, many 'exceptions' land on the next day after deadline, stress that can be avoided

Enhancements Slush will simply declare the week before Freeze as "this week your KEP should _really_ be ready if you want it in, it needs XYZ", and have a good 10 days for everyone to see and prepare until freeze. It shouldn't be viewed as another Ticket Police Event that partially filters things out - Policing only increases stress in all participants.
At this point I would also suggest to push the Freeze date to Week 5, with Week 4 being start of "Slush".

This will give all involved parties more time to sort everything out, ideally arrange review timeslots beforehand, synchronize back and forth and so on. To give time to delegate reviewing to others if overburdened, time to change things post-review, time and space.

It's good to mention that Slush existed as Code Slush at Week 9 and was thankfully removed - it was adding more problems instead of solving as everything was re-labeled twice without much reason - singleton PRs are way more flexible than Enhancements.

Ultimately it's not a big change, but it will likely easen the mentality and "urgency" around the deadline.

You received this message because you are subscribed to the Google Groups "Kubernetes Release Team" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-releas...@googlegroups.com.

To post to this group, send email to kubernetes-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-release-team/CAHROWxQH4i8DM9wk%2B9WHfe2HNWJDe8BvSh7tiPkgmQCOW6cMLA%40mail.gmail.com.

[Tim Hockin]

Let me start by apologizing. I was a little flustered when I wrote
the original note. I feel a lot of personal responsibility and I was
disappointed that I was not going to be able to deliver.

Some of this is clearly my fault. I have too many opinions on too
many things. I find myself weighing in on KEPs from a multitude of
SIGs from experienced and new contributors. I did not adequately
allocate time over the last month, and I did not plan for the
last-second crush. Mea culpa.

Some of this is procedural. As folks are discussing here, there are
probably things we can do better to make things more obvious and
incremental. Some of the feedback I had for people this week should
have been done much earlier.

I want to be really clear - my concern is NOT that people were
"bugging" me. I generally encourage people to reach out to me if I
lose track of something. My concern was that some of these pings
could have happened earlier so I could spread them out more. It was
the last-second nature of it all (and timezones make it even harder)
where folks were (rightly) disappointed that I did not get to their
KEP in the last 4 weeks, and then when I did get to it, I had material
feedback. Now rinse and repeat, and see how many times you can review
the same KEP in a day :)

This was exacerbated by my "other" responsibilities which I probably
should have postponed in anticipation of this week. Next cycle I am
going to clear my calendar the week before deadline.

On API reviews: I 100% agree that a KEP doesn't need to have a fully
polished API. There's an API review process. KEP should be focused
on the idea, the boundaries and semantics, and the general expression
of that idea. It's probably bad if the final API looks nothing like
the KEP, but not plan survives contact with the enemy. That said, I
had a few proposals this week that were wildly off-target API wise.

My last topic: I read KEPs this week that I had never seen before. I
don't know how to better stay abreast of all of the things in flight
to know where to weigh in. There were a few things that clearly
needed high-level architectural feedback that they had not received
yet, and some that probably warrant mutliple architecture reviewers
paying attention to.

TL;DR I will take steps to make this hurt less on my side. As a
community I think we have room for improvement. As
architectural-leaders we need to be vigilant and we need to stick our
noses into places even when nobody asked us.

[Clayton Coleman]

I want to highlight something here that I think gets to a level of
responsibility and burden that I’m reasonably sure other reviewers
feel.

Kube is deployed in a huge range of environments, from the hobbyist
raspberry pi clusters all the way to very serious production clusters
running mission critical software where people’s lives can be
impacted.

We as humans and engineers (usually the same thing) have an obligation
to provide a certain level of responsibility to our consumers that we
do not regress or surprise them. We aren’t responsible for everything
our consumers do, but we have to act with a certain amount of caution
and due diligence. That caution has to be balanced against our very
natural desire for features, clean ups, refractors, and bug fixes.

If a security vulnerability slips into Kube, and as a result down the
road 100m people have their financial records stolen, can we look at
ourselves and say we did what we reasonably could to ensure that
didn’t happen?

If a production outage due to an ill considered change causes a
medical device monitoring system to fail and someone to die, could we
look at the series of changes that led to that change and say we
adequately balanced risk with reward?

We are *not* responsible for all software and the consequences of its use.

But as an open source project we are responsible for a certain amount
of careful consideration about the changes we make. Kube is foremost
a software project for enabling others to succeed. We should always,
always, *ALWAYS* be willing to slow down, take some time, and deliver
it tomorrow.

I would ask that we continue (I see us doing a pretty good job of this
so far, but reflection is good) to pace the rate of change to a level
that

1. Our contributors can tolerate
2. Our reviewers can manage
3. Our consumers can absorb
4. Our community can manage

The most important job of an engineer is to say “no, we need to think
more about this”. If we see signs that we aren’t able to do that, I’d
request that we all create space in how we execute to allow that to
continue to happen.

> To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-sig-architecture/CAO_RewYm2U9pdN8DGBfM1fw8S1NdR2KvYSRMRio_g0167XPM3g%40mail.gmail.com.

[Brendan Burns]

I think that Clayton raises a good point. Something that is missing from the current process is both a measure of the risk ("how scary is the kep") and the priority ("how important is the kep").

In the absence of these the priority ordering tends to be "how noisy is the kep?" Which is clearly bad.

Maybe a missing piece is some sorry if priority ordering on all keps. This could be a pretty lightweight process that would ensure that we focus the right energy and diligence in the right places.

--brendan

---

From: kubernetes-...@googlegroups.com <kubernetes-...@googlegroups.com> on behalf of Clayton Coleman <ccol...@redhat.com>
Sent: Thursday, May 2, 2019 9:14:28 AM
To: Tim Hockin
Cc: Niko Penteridis; Derek Carr; Andrew Kim; Brendan Burns; Davanum Srinivas; Kubernetes Release Team; Stephen Augustus; kubernetes-sig-architecture; kubernetes-sig-leads; kubernet...@googlegroups.com; kubernetes-sig-release

Subject: Re: Deadlines are horrible

>>>>> TSC and SIG Cluster Lifecycle set an excellent example here: https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fkubernetes%2Fcommunity%2Fblob%2Fmaster%2Fsig-cluster-lifecycle%2Fgrooming.md&amp;data=01%7C01%7Cbburns%40microsoft.com%7C70763e3e8cf6481bbe0d08d6cf194427%7C72f988bf86f141af91ab2d7cd011db47%7C1&amp;sdata=sL2GqHL16FZQkkEBVq2HNHxj4hanal5hT2b7suzOYY8%3D&amp;reserved=0

>>>>>>>> To view this discussion on the web visit https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.google.com%2Fd%2Fmsgid%2Fkubernetes-sig-release%2FCAO_RewZkYLfBpiUfXSB%253Do1mTh0zpRqA3Nk1fggx99JB2B%252BJDhg%2540mail.gmail.com&amp;data=01%7C01%7Cbburns%40microsoft.com%7C70763e3e8cf6481bbe0d08d6cf194427%7C72f988bf86f141af91ab2d7cd011db47%7C1&amp;sdata=9269%2FSIfQNTzAdb2%2BfCqi6WXV6k4L7QTxW2UhA3WcVY%3D&amp;reserved=0.
>>>>>>>> For more options, visit https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.google.com%2Fd%2Foptout&amp;data=01%7C01%7Cbburns%40microsoft.com%7C70763e3e8cf6481bbe0d08d6cf194427%7C72f988bf86f141af91ab2d7cd011db47%7C1&amp;sdata=5VLzOWbXzYzQtQRxMjmTXpzOt3ILj1wID5%2BR1D9ex5o%3D&amp;reserved=0.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Davanum Srinivas :: https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftwitter.com%2Fdims&amp;data=01%7C01%7Cbburns%40microsoft.com%7C70763e3e8cf6481bbe0d08d6cf194427%7C72f988bf86f141af91ab2d7cd011db47%7C1&amp;sdata=CxkoxOdMn%2FKHMynHS%2FU6lVSCj7QW2saS8MrJPNo9G4Y%3D&amp;reserved=0

>>>>>>
>>>>>> --
>>>>>> You received this message because you are subscribed to the Google Groups "kubernetes-sig-release" group.
>>>>>> To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-re...@googlegroups.com.
>>>>>> To post to this group, send email to kubernetes-...@googlegroups.com.

>>>>>> To view this discussion on the web visit https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.google.com%2Fd%2Fmsgid%2Fkubernetes-sig-release%2F08183e9c-7496-4189-a4fc-24b0fc0e1746%2540googlegroups.com&amp;data=01%7C01%7Cbburns%40microsoft.com%7C70763e3e8cf6481bbe0d08d6cf194427%7C72f988bf86f141af91ab2d7cd011db47%7C1&amp;sdata=pswUPxp%2FIJy5ruVXU8QBU6kmsFmEOg4dtB5RFsEo3AY%3D&amp;reserved=0.
>>>>>> For more options, visit https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.google.com%2Fd%2Foptout&amp;data=01%7C01%7Cbburns%40microsoft.com%7C70763e3e8cf6481bbe0d08d6cf194427%7C72f988bf86f141af91ab2d7cd011db47%7C1&amp;sdata=5VLzOWbXzYzQtQRxMjmTXpzOt3ILj1wID5%2BR1D9ex5o%3D&amp;reserved=0.

>>>>>
>>>>> --
>>>>> You received this message because you are subscribed to the Google Groups "kubernetes-sig-architecture" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-arch...@googlegroups.com.
>>>>> To post to this group, send email to kubernetes-si...@googlegroups.com.

>>>>> To view this discussion on the web visit https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.google.com%2Fd%2Fmsgid%2Fkubernetes-sig-architecture%2FCAFQm5yR-xv3gJ%253DBWyJBUSu3NLJqqprz8v-erkn43S4%253DG9-wTwQ%2540mail.gmail.com&amp;data=01%7C01%7Cbburns%40microsoft.com%7C70763e3e8cf6481bbe0d08d6cf194427%7C72f988bf86f141af91ab2d7cd011db47%7C1&amp;sdata=j5e3IiyYZNBnHGxIdf%2F895JuaPET4pe265y6AHhwMIY%3D&amp;reserved=0.
>>>>> For more options, visit https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.google.com%2Fd%2Foptout&amp;data=01%7C01%7Cbburns%40microsoft.com%7C70763e3e8cf6481bbe0d08d6cf194427%7C72f988bf86f141af91ab2d7cd011db47%7C1&amp;sdata=5VLzOWbXzYzQtQRxMjmTXpzOt3ILj1wID5%2BR1D9ex5o%3D&amp;reserved=0.

>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google Groups "kubernetes-sig-architecture" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-arch...@googlegroups.com.
>>>> To post to this group, send email to kubernetes-si...@googlegroups.com.

>>>> To view this discussion on the web visit https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.google.com%2Fd%2Fmsgid%2Fkubernetes-sig-architecture%2FCABc050F17GTYcN%253DOvkTVAoytBqVcA%253Dpzzm3hsK8dKtLM9iWhSg%2540mail.gmail.com&amp;data=01%7C01%7Cbburns%40microsoft.com%7C70763e3e8cf6481bbe0d08d6cf194427%7C72f988bf86f141af91ab2d7cd011db47%7C1&amp;sdata=zPY22nYQfUg9gSXDcuEPFQMrTADtj%2BZEm2XaaWyiAUw%3D&amp;reserved=0.
>>>> For more options, visit https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.google.com%2Fd%2Foptout&amp;data=01%7C01%7Cbburns%40microsoft.com%7C70763e3e8cf6481bbe0d08d6cf194427%7C72f988bf86f141af91ab2d7cd011db47%7C1&amp;sdata=5VLzOWbXzYzQtQRxMjmTXpzOt3ILj1wID5%2BR1D9ex5o%3D&amp;reserved=0.

>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups "Kubernetes Release Team" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-releas...@googlegroups.com.
>>> To post to this group, send email to kubernetes-...@googlegroups.com.

>>> To view this discussion on the web visit https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.google.com%2Fd%2Fmsgid%2Fkubernetes-release-team%2FCAHROWxQH4i8DM9wk%252B9WHfe2HNWJDe8BvSh7tiPkgmQCOW6cMLA%2540mail.gmail.com&amp;data=01%7C01%7Cbburns%40microsoft.com%7C70763e3e8cf6481bbe0d08d6cf194427%7C72f988bf86f141af91ab2d7cd011db47%7C1&amp;sdata=cJF%2F8JzXkZZ%2BYbRCtw%2Fay23K%2F0XhCtOGPOEYpXoR%2F8M%3D&amp;reserved=0.
>>> For more options, visit https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.google.com%2Fd%2Foptout&amp;data=01%7C01%7Cbburns%40microsoft.com%7C70763e3e8cf6481bbe0d08d6cf194427%7C72f988bf86f141af91ab2d7cd011db47%7C1&amp;sdata=5VLzOWbXzYzQtQRxMjmTXpzOt3ILj1wID5%2BR1D9ex5o%3D&amp;reserved=0.

>
> --
> You received this message because you are subscribed to the Google Groups "kubernetes-sig-architecture" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-arch...@googlegroups.com.
> To post to this group, send email to kubernetes-si...@googlegroups.com.

> To view this discussion on the web visit https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.google.com%2Fd%2Fmsgid%2Fkubernetes-sig-architecture%2FCAO_RewYm2U9pdN8DGBfM1fw8S1NdR2KvYSRMRio_g0167XPM3g%2540mail.gmail.com&amp;data=01%7C01%7Cbburns%40microsoft.com%7C70763e3e8cf6481bbe0d08d6cf194427%7C72f988bf86f141af91ab2d7cd011db47%7C1&amp;sdata=K3n9j5QUv3HLcrf1eHl3P%2Bu6sRCeEjtqR45jYiuOgZg%3D&amp;reserved=0.
> For more options, visit https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.google.com%2Fd%2Foptout&amp;data=01%7C01%7Cbburns%40microsoft.com%7C70763e3e8cf6481bbe0d08d6cf194427%7C72f988bf86f141af91ab2d7cd011db47%7C1&amp;sdata=5VLzOWbXzYzQtQRxMjmTXpzOt3ILj1wID5%2BR1D9ex5o%3D&amp;reserved=0.

--
You received this message because you are subscribed to a topic in the Google Groups "kubernetes-sig-release" group.
To unsubscribe from this topic, visit https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.google.com%2Fd%2Ftopic%2Fkubernetes-sig-release%2FdGVBrlkOXQo%2Funsubscribe&amp;data=01%7C01%7Cbburns%40microsoft.com%7C70763e3e8cf6481bbe0d08d6cf194427%7C72f988bf86f141af91ab2d7cd011db47%7C1&amp;sdata=Kol65ujBCnv%2Bzm6ljZRBwleQJBrhGP%2BjchmH%2BTMxZy0%3D&amp;reserved=0.
To unsubscribe from this group and all its topics, send an email to kubernetes-sig-re...@googlegroups.com.

To post to this group, send email to kubernetes-...@googlegroups.com.

To view this discussion on the web visit https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.google.com%2Fd%2Fmsgid%2Fkubernetes-sig-release%2FCAH16ShJw24nMzJ2ckBWN-eesds9hYPc29vpR%252B5C496vEi61XyQ%2540mail.gmail.com&amp;data=01%7C01%7Cbburns%40microsoft.com%7C70763e3e8cf6481bbe0d08d6cf194427%7C72f988bf86f141af91ab2d7cd011db47%7C1&amp;sdata=6Mxz%2FbXexJOKMwE%2F%2FJsxrOtPL79j66ois8rlRPnv80M%3D&amp;reserved=0.
For more options, visit https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.google.com%2Fd%2Foptout&amp;data=01%7C01%7Cbburns%40microsoft.com%7C70763e3e8cf6481bbe0d08d6cf194427%7C72f988bf86f141af91ab2d7cd011db47%7C1&amp;sdata=5VLzOWbXzYzQtQRxMjmTXpzOt3ILj1wID5%2BR1D9ex5o%3D&amp;reserved=0.

[Tim Hockin]

Agree with all that Clayton says here. The part that I struggle with
is that I feel a sense of "injustice" when a well-intentioned
contributor has actually followed the written rules and then gets
torpedoed at the last second.

I have proposed before, half jokingly, that sig-arch do a
turbo-readthrough of every KEP (yes, *every*) and decide "do we care?"
and "who from sig-arch is going to act as arch reviewer". I would
personally find it useful to boil KEPs down to 2-3 sentences so I'd
know which ones I intend to track, rather than getting looped in at
the last second.

[Andrew Kim]

I have proposed before, half jokingly, that sig-arch do a
turbo-readthrough of every KEP (yes, *every*) and decide "do we care?"
and "who from sig-arch is going to act as arch reviewer".  I would
personally find it useful to boil KEPs down to 2-3 sentences so I'd
know which ones I intend to track, rather than getting looped in at
the last second.

I think this was what I was trying to get at earlier s/API Approver/Arch Approver . IMHO this would be tremendously valuable. I think many SIG chairs/TLs would feel empowered to be more active in the KEP process if there's an architecture reviewer/approver (different from SIG approver) explicitly assigned to it. I wouldn't really expect them to follow the KEP in much detail, but a simple "this is the right/wrong direction" every once in a while goes a REALLY long way.

[Matt Farina]

What a great conversation we need to have.

I like what Clayton had to say about thinking things through. K8s isn't a startup project trying to get off the ground any longer. The features people MUST have to run large scale things for a huge set of cases are there. Now we can think through what we do next and who/how it impacts the things.

I also appreciate the problem Tim ran into. Waiting till the last minute minds me of people in school waiting till the last minute on a project and running to their teacher at office hours with questions.

Some ideas....

1. Ask SIGs to do regular KEP triage. Some of them do issue triage. KEP triage would be useful, too If it's a regular activity it may spread the load over the course of the release cycle
2. I wonder if a KEP triage role could exist for some SIGs. Someone to look at
   
3. It would be great to have a project board tracking KEP changes/PRs in flight. A simple filter by SIG could provide a board to show/discuss in a meeting
4. How about a KEP tracking board that tracks the progress of the ones we want implementable/implemented in a release. I thought we had one but I couldn't find it. This might get SIGs (like SIG Arch) to know which subset to focus on first.
   
5. It maybe time to level up more people in their ability to review KEPs.
6. Clone Tim (Multiplicity anyone?)
   

- Matt

You received this message because you are subscribed to the Google Groups "kubernetes-sig-leads" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-l...@googlegroups.com.
To post to this group, send email to kubernetes...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-sig-leads/CABc050HvpoZ72r8-oPGFuO%2BzV%2BxiOAYjN_5L33azHc0BrOo7MQ%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Matt Farina

mattfarina.com

Go in Practice - A book of Recipes for the Go programming language.

Code Engineered - A blog on cloud, web, and software development.

[Linus Arver]

Would it make sense to have some sort of crude voting system for the KEPs? It would make the prioritization self-evident. E.g., have a column for "votes" in each KEP.

I suppose the hard part would be figuring out who can vote on KEPs, and how long the voting window lasts.

To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-sig-release/CAMPAG2o7PohyDVUJD3w_HcdK7gR%2Ba3-Bs08ErcuxLiLVfDaBDw%40mail.gmail.com.

[Brian Grant]

On Thu, May 2, 2019 at 11:21 AM Matt Farina <matt....@gmail.com> wrote:

What a great conversation we need to have.

I like what Clayton had to say about thinking things through. K8s isn't a startup project trying to get off the ground any longer. The features people MUST have to run large scale things for a huge set of cases are there. Now we can think through what we do next and who/how it impacts the things.

I also appreciate the problem Tim ran into. Waiting till the last minute minds me of people in school waiting till the last minute on a project and running to their teacher at office hours with questions.

Some ideas....

1. Ask SIGs to do regular KEP triage. Some of them do issue triage. KEP triage would be useful, too If it's a regular activity it may spread the load over the course of the release cycle
2. I wonder if a KEP triage role could exist for some SIGs. Someone to look at
   
3. It would be great to have a project board tracking KEP changes/PRs in flight. A simple filter by SIG could provide a board to show/discuss in a meeting
4. How about a KEP tracking board that tracks the progress of the ones we want implementable/implemented in a release. I thought we had one but I couldn't find it. This might get SIGs (like SIG Arch) to know which subset to focus on first.
   

Also a schedule with a bounded number of review slots per week, spread over the entire calendar (independent of release deadlines) 

To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-sig-architecture/CAMPAG2o7PohyDVUJD3w_HcdK7gR%2Ba3-Bs08ErcuxLiLVfDaBDw%40mail.gmail.com.

[Tim Hockin]

I think the calendar is something I will have to do myself and just be more frank with people that they didn't reserve a slot.

That said, I *want* to see other KEPs from other SIGs.  There's interesting stuff happening, and as always I reserve the right to have an opinion, even about things I don't really understand :)

[Mike Spreitzer]

I hope nobody is flying airplanes based on kube, but anyone who hasn't yet should read up on what's going on with the 737 Max.  There too the main message seems, to me, to be that letting schedule override good engineering is a recipe for disaster.

Thanks,
Mike

[Daniel Smith]

I'm quite late to this thread, but I am in the same boat as Tim. The deadline snuck up on me (I didn't even realize there was a deadline that somehow included KEPs), the week collided with Other Work Things, and I caught some sort of terrible cold and was pretty ineffectual all week. I think there are still a couple of KEPs that I want to go beg for an exception on.

SIG API Machinery actually has done a KEP roundup at most of our meetings this quarter, so it's not like we didn't know about the KEPs. One could say that this was my own scheduling failure, which is pretty fair. But it still seems like something isn't right if a deadline like this DoSes multiple approvers. There's got to be a better process--I am not sure what, though. I understand the importance of arranging for docs etc etc, but this deadline still felt arbitrary and capricious (to be clear, this is a fact about me and not necessarily a fact about the deadline).

Anyway, probably none of this is actionable. I mostly just wanted it to be clear that this was rough for a lot of folks.

P.S. My standard rant here is that if the train left the station every two weeks rather than every 3 months, we wouldn't have this issue. (To be fair, we would have plenty of other, new issues.)

--
You received this message because you are subscribed to the Google Groups "Kubernetes SIG PM" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig...@googlegroups.com.
To post to this group, send email to kubernet...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-sig-pm/OFEFAF6F6A.1F54DD6E-ON852583F0.005AE039-852583F0.005B2772%40notes.na.collabserv.com.

[Matt Farina]

Should we have a calendar everyone can subscribe to with alerts a week before any deadline?

Just thinking out loud...

You received this message because you are subscribed to the Google Groups "kubernetes-sig-leads" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-l...@googlegroups.com.
To post to this group, send email to kubernetes...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-sig-leads/CAB_J3bYLvedZRgPADrnf8okYtLgYLckeLzdKqC9jF1ZsRO%2BAdg%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

[Stephen Augustus]

The community calendar, perhaps?

I believe there may be a release event or two that already go up there.

[Tim Hockin]

A calendar is not a full solution, but could help. If it is too
noisy, though, it will get ignored.

[Paris Pittman]

community calendar

ical version

To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-sig-leads/CAO_RewaFx480M4%3DuDhvbemXjCsroCkQQK-2QJCQaO0aychDLVw%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Paris Pittman Kubernetes Community Open Source Strategy, Google Cloud 345 Spear Street, San Francisco, 94105

[Tim Hockin]

Yeah, that's WAY too noisy

[Stephen Augustus]

We can make it part of the SIG Release calendar instead (which only has our fortnightly meeting on it at the moment).

[Quinton Hoole]

Great thread! Very civilized brainstorm. Congrats to all (Especially to Tim for setting the right tone to begin with).

Q

Q

You received this message because you are subscribed to the Google Groups "kubernetes-sig-architecture" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-arch...@googlegroups.com.
To post to this group, send email to kubernetes-si...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-sig-architecture/CAFQm5yThentX05iyfpEEG8BT7WoKPLx8%3DtKdzQS-915wxOfjKg%40mail.gmail.com.

[Matt Farina]

When I was thinking of a calendar and notifications to help with this problem I was thinking of one that had dates like code freeze, enhancements freeze, and so forth. It would reflect a limited subset of the release cycle at https://github.com/kubernetes/sig-release/tree/master/releases/release-1.15. The important part is notifications a week or so before and dates on calendars so people can see it rather than go out and remember to check it. The idea is to give people a week to scramble instead of a day (if they even realized that).

I was not thinking of all the things going on in the community.

[Stephen Augustus]

Right. We can put key dates on the SIG Release calendar[1] moving forward (as mentioned below).

I'm short on time to do that at the moment, but that's where they'll be. You'll need to be a member of the SIG Release Google Group[2] for access.

The current timeline[3] is available on GitHub and I've opened an issue[4] to update the calendar, in case someone can get to it before me.

-- Stephen

[1] https://calendar.google.com/calendar/embed?src=kipmnllvl17vl9m98jen6ujcrs%40group.calendar.google.com&ctz=America%2FNew_York

[2] https://groups.google.com/forum/#!forum/kubernetes-sig-release

[3] https://github.com/kubernetes/sig-release/tree/master/releases/release-1.15#timeline

[4] https://github.com/kubernetes/sig-release/issues/617

[Aaron Crickenberger]

Creation of a calendar dedicated to the release is documented here, maybe we forgot to do it this time? https://github.com/kubernetes/sig-release/tree/master/release-team/role-handbooks/release-team-lead#week-2

- aaron

You received this message because you are subscribed to the Google Groups "Kubernetes Release Team" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-releas...@googlegroups.com.
To post to this group, send email to kubernetes-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-release-team/CAFQm5ySa9pWgdrkA6zRvfn65T%3DVQBVJeCDP7y8F1dwNKup0mTQ%40mail.gmail.com.

[Stephen Augustus]

Hmmm, I think people would be more likely to subscribe to the SIG Release calendar than a release-specific one. Maybe we should change those instructions?

It'd also obviate the need to remember to join a calendar every cycle, ensures that the RT subproject owners + Chairs always have appropriate access to the calendar, and allows you to view all historical key dates across cycles.

[Daniel Smith]

Speaking for myself, I'm vastly more likely to subscribe once to a single release calendar than multiple times to per-release calendars.

I would recommend making a separate calendar from SIG Release's existing calendar, though. The audience for these dates is much larger than SIG Release; even if today there's only one non-relevant thing on there, that may not be true in the future.

[Claire Laurence]

Hey Folks,

There is lots of great feedback here!  I've added some of the feedback from this thread to the 1.15 Release Retro doc  that is specific to the release process (leaving out feedback on the KEP review process as it's somewhat out of scope of the 1.15 release).  Feel free to add additional commentary that you feel is relevant to be discussed during the 1.15 Release Retro.

Regarding the 1.15 calendar if it would be helpful for me to move the upcoming milestones to a central SIG release calendar I'm more than happy to do that - or we can discuss it during the 1.15 retro and consider it something we might want to change in the 1.16 cycle.  

Best,

Claire 

To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-release-team/CAB_J3bZB4Ef2gzjmJbxAYKyoB6vftM_oqi01b5Xoay1HhLDQtQ%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Claire Laurence

Technical Program Manager

Cloud R&D | New York