RFC: Issue Triage Labels & Workflow Change

[Bob Killen]

Hey there all,

We'd like to introduce a small workflow change to the kubernetes/kubernetes repo that should hopefully help groups with their issue triage process. This was previously discussed back in April[1], but we've collected more feedback since then and believe we have a good solution. You can read more about it on kubernetes/enhancements#1930[2]. There is a lazy consensus date of 2020-08-28. For your convenience a brief overview is included below:

New Workflow

A new required label called needs-triage will be applied automatically to issues created within the kubernetes/kubernetes repository similar to the current needs-sig or needs-kind labels. This serves as a boolean signal to community group members that the issue has not yet been triaged.

This label can be removed by applying one of the triage labels[3] or a new triage/accepted label that signifies the issue has enough information and can be worked on.

This should make it easier for groups to do their own triage sessions and mark off which issues have been evaluated without having to resort to some other form of tracking.

Re-categorize the triage/support label

The label triage/support will become kind/support. kind/* better reflects the class or or type of issue.

If you have any questions or concerns, please try and get them in by 2020-08-28. We would like to roll this change out for use with the 1.20 release cycle.

Thanks!

- Bob Killen, Stephen Augustus, Nikhita Raghunath

[1]: https://groups.google.com/d/msg/kubernetes-dev/tJ5Y_av0Bao/kbAWYBfHBAAJ

[2]: https://github.com/kubernetes/enhancements/pull/1930

[3]: https://github.com/kubernetes/kubernetes/labels?q=triage

[Tim Hockin]

YAY!!!!

As we get close, nits:

Can we document somewhere that ONLY THE SIG THAT IS ACCEPTING THE
ISSUE may apply triage/accepted? SIGs are going to use needs-triage
as a primary signal, and we need that to not get obscured by
well-meaning people doing triage outside of SIG context.

On triage labels:

> triage/support "Indicates an issue that is a support question."

You say this will move to kind/support - but I thought we all agreed
that support requests should be closed and redirected to proper
support channels?

> triage/unresolved "Indicates an issue that can not or will not be resolved."

This one doesn't make sense to me. "Unresolved" and "unresolvable"
are not the same thing. Do we even need this? Would it be better as
"will not fix"? Should all such issues be closed?

> triage/duplicate "Indicates an issue is a duplicate of other open issue."

I assume all such issues should be closed?

> triage/needs-information "Indicates an issue needs more information
in order to work on it."

This seems reasonable on the surface, but I am not surte. What is the
expected protocol? Is it mutually exclusive with needs-triage? If I
want to see all un-triaged issues I need to query for needs-triage OR
triage/needs-information? You said "a boolean signal" - can we make
"needs info" orthogonal to "needs-triage"? IOW, not
"triage/needs-info" but something else.

> triage/not-reproducible "Indicates an issue can not be reproduced as described."

Should all such issues be closed, or left open?

For all of the triage/* that should be closed, can we a) document (in
the label description?) and maybe b) enforce (bot?) that?

If someone re-opens a closed issue, will the bot re-apply the
needs-triage? It seems like it should..

> --
> You received this message because you are subscribed to the Google Groups "Kubernetes developer/contributor discussion" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-de...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-dev/CAJ18wa_%3DRLZw6xycCpmt1ArYtV5vBzQ4q2u5v%2Bza-%2BnhObj3fg%40mail.gmail.com.

[Daniel Smith]

This sounds amazing. Will be a huge help running our bug triage sessions!

Can we use this for PRs, too? We triage both.

To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-dev/CAO_Rewb%3DkV9SyTBdWzEfJbHQQn%3DSNuyNzfFUwqi0QWGcJr3F-A%40mail.gmail.com.

[Benjamin Elder]

Regarding kind/support:

These labels exist for *all repos in both github organizations*, while Kubernetes/Kubernetes may agree to close support questions (in practice I'm seeing them responded to lately ...), other smaller repos may actually intend to handle support question issues without the overhead of yet-another-site to track.

Regardless of closing them, kind/ (categories of issue) is a more accurate label, distinguishing the issue from a bug, failing-test, feature request, etc.

> This one doesn't make sense to me.  "Unresolved" and "unresolvable"
are not the same thing.  Do we even need this?  Would it be better as
"will not fix"?  Should all such issues be closed?

+1 ...

> > triage/duplicate "Indicates an issue is a duplicate of other open issue."
>
> I assume all such issues should be closed?

That's how it's being used currently. Tag as duplicate then close with a link to the original. (this label already exists)

> This seems reasonable on the surface, but I am not surte.  What is the
expected protocol?  Is it mutually exclusive with needs-triage? If I
want to see all un-triaged issues I need to query for needs-triage OR
triage/needs-information?  You said "a boolean signal" - can we make
"needs info" orthogonal to "needs-triage"?  IOW, not
"triage/needs-info" but something else.

Accepted / no longer needs triage, but awaiting information to move further? (So yes, exclusive?)
Somewhere it's useful to have a signal to would-be bug fixers that an issue sounds real and is accepted by a SIG / repo, but needs more information before work can be done.

This label also already exists.

> > triage/not-reproducible "Indicates an issue can not be reproduced as described."
>
> Should all such issues be closed, or left open?

This is a nuanced alternative to "needs-information" IMHO. OP has provided information in detail, but for whatever reason we're unable to reproduce it.
This is another sign that this issue is more or less valid, but currently blocked.

To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-dev/CAB_J3bZsq8E7v_PmYgkDQvquSEj3D35syCtzJWP_S4XtG-UEVw%40mail.gmail.com.

[Tim Hockin]

On Tue, Aug 18, 2020 at 11:39 AM Benjamin Elder <benth...@google.com> wrote:
>
> Regarding kind/support:
>
> These labels exist for *all repos in both github organizations*, while Kubernetes/Kubernetes may agree to close support questions (in practice I'm seeing them responded to lately ...), other smaller repos may actually intend to handle support question issues without the overhead of yet-another-site to track.
>
> Regardless of closing them, kind/ (categories of issue) is a more accurate label, distinguishing the issue from a bug, failing-test, feature request, etc.

ACK, I was myopic.

> > This seems reasonable on the surface, but I am not surte. What is the
> expected protocol? Is it mutually exclusive with needs-triage? If I
> want to see all un-triaged issues I need to query for needs-triage OR
> triage/needs-information? You said "a boolean signal" - can we make
> "needs info" orthogonal to "needs-triage"? IOW, not
> "triage/needs-info" but something else.
>
> Accepted / no longer needs triage, but awaiting information to move further? (So yes, exclusive?)

I don't know how to parse that, sorry. If I set "triage/needs-info",
it will clear "needs-triage". But (in my mind) an issue that "needs
info", by definition, can not be fully triaged. "triage/accepted"
means "We know that this is a (bug|feature)" and you can't REALLY know
that if you need more info.

> Somewhere it's useful to have a signal to would-be bug fixers that an issue sounds real and is accepted by a SIG / repo, but needs more information before work can be done.

I'd like to be able to say "this needs triage" *and* "it needs more
information". If we need "more info" it should not be under triage/

> This label also already exists.

ehhh, we can rename it ?

> > > triage/not-reproducible "Indicates an issue can not be reproduced as described."
> >
> > Should all such issues be closed, or left open?
>
> This is a nuanced alternative to "needs-information" IMHO. OP has provided information in detail, but for whatever reason we're unable to reproduce it.
> This is another sign that this issue is more or less valid, but currently blocked.

I'm fine with this, but I worry about the state changes. If I mark an
issue not-reproducible, then someone adds info to it, what I really
want is for it to come back to needs-triage (which, IMO, should happen
on any re-opened issue).

Tim

[Tim Hockin]

On Tue, Aug 18, 2020 at 12:04 PM Tim Hockin <tho...@google.com> wrote:
>
> On Tue, Aug 18, 2020 at 11:39 AM Benjamin Elder <benth...@google.com> wrote:
> >
> > Regarding kind/support:
> >
> > These labels exist for *all repos in both github organizations*, while Kubernetes/Kubernetes may agree to close support questions (in practice I'm seeing them responded to lately ...), other smaller repos may actually intend to handle support question issues without the overhead of yet-another-site to track.
> >
> > Regardless of closing them, kind/ (categories of issue) is a more accurate label, distinguishing the issue from a bug, failing-test, feature request, etc.
>
> ACK, I was myopic.
>
> > > This seems reasonable on the surface, but I am not surte. What is the
> > expected protocol? Is it mutually exclusive with needs-triage? If I
> > want to see all un-triaged issues I need to query for needs-triage OR
> > triage/needs-information? You said "a boolean signal" - can we make
> > "needs info" orthogonal to "needs-triage"? IOW, not
> > "triage/needs-info" but something else.
> >
> > Accepted / no longer needs triage, but awaiting information to move further? (So yes, exclusive?)
>
> I don't know how to parse that, sorry. If I set "triage/needs-info",
> it will clear "needs-triage". But (in my mind) an issue that "needs
> info", by definition, can not be fully triaged. "triage/accepted"
> means "We know that this is a (bug|feature)" and you can't REALLY know
> that if you need more info.
>
> > Somewhere it's useful to have a signal to would-be bug fixers that an issue sounds real and is accepted by a SIG / repo, but needs more information before work can be done.
>
> I'd like to be able to say "this needs triage" *and* "it needs more
> information". If we need "more info" it should not be under triage/

What I am really driving at is that there should be ONE unambiguous,
unchanging way to ask "what does my SIG need to look at this week". I
think that should be "sig/foo AND needs-triage". If we take
needs-triage off for ANY reason other than "accepted" I am not going
to find it. If you change the query to "sig/foo AND (needs-triage OR
triage/needs-info)" then you have opened the door to more triage/*
labels that still are not fully triaged.

[Antoine Pelisse]

Maybe you can assign someone to follow-up on things that need more info? Possibly re-triaging later, and/or re-assign to a more suitable owner.

To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-dev/CAO_RewZXJ-yZgFOB83s1uRspwr0xJcP%2B3MXKbLxLi__0aDhawQ%40mail.gmail.com.

[Tim Hockin]

Without a backstop, that just isn't working. We have to revisit them
frequently, which is why I want it TOTALLY unambiguous. :)

[Bob Killen]

On Tue, Aug 18, 2020 at 1:26 PM Daniel Smith <dbs...@google.com> wrote:

This sounds amazing. Will be a huge help running our bug triage sessions!

Can we use this for PRs, too? We triage both.

It's currently only scoped for issues, but we can potentially open it up to PRs.

[Federico Bongiovanni]

Having it consistently applied for PRs too would be huge! 

To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-dev/CAJ18wa_dzKzAHpC9AOwhFb6FnYYt4dg_uKvVxVb3wF0q%2BH%3D0AA%40mail.gmail.com.

[Bob Killen]

The issue is that other groups do use the triage class of labels. Now, most of those labels aren't widely used in k/k[1] (unresolved is almost entirely athenabot). We could update the verbiage in the KEP and our triage documentation that for k/k we should stick to using needs-triage like a boolean flag, and apply the other triage labels only when the issue is to be closed. It won't serve as a hard blocker, and it does mean it is possible for some things to fall through the cracks if a label is mis-applied. However, it should cover the majority of scenarios while still leaving room for other groups and other repos to use the triage class of labels to suit their various workflows.

[1]: https://github.com/kubernetes/kubernetes/labels?q=triage

[Tim Hockin]

I don't want to be a picky jerk, so I will happily use whatever works
for the majority. I just ask that we keep the number of triage/*
labels low and relatively constant, so queries don't get too wonky.