Is the "to be reviewed" tag really useful?

[Antonin Delpeuch (lists)]

Hi all,

As part of our issue templates on the main repository, we add the "to be
reviewed" tag to all new issues.

I have the feeling that we do not really use this tag in our processes,
after the issue is created. Sometimes I take the time to remove it from
issues which look sensible and well-scoped to me, but I wonder if it is
really useful.

It does make sense not to encourage people to submit pull requests on
issues which could be ill informed, but the noise/signal ratio with this
tag is pretty low I think.

I would propose to remove this tag altogether. What do you think?

Antonin

[Thad Guidry]

yes remove, I was just thinking about that!

Thad

https://www.linkedin.com/in/thadguidry/

https://calendly.com/thadguidry/

--
You received this message because you are subscribed to the Google Groups "OpenRefine Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openrefine-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openrefine-dev/5056b0fd-e8be-0f24-4297-1e7c90649e26%40antonin.delpeuch.eu.

[Tom Morris]

The original purpose of this tag was to provide a clear indicator of which issues need triage. In my experience, bug triage is a useful (and usually necessary) step in the issue intake process. It's a relatively common pattern employed in projects, although some bug trackers support it through mechanisms other than tags. An example of another Github project that uses tags for this mechanism is OpenLibrary, where the tag is called "Needs: Triage."

Of course, if it's not being used, there's no sense in having it, but another approach might be to find a way to triage bugs that doesn't depend on the project leader. I think this is largely an administrative, not technical, function which looks at things like:

• Is the issue a non-duplicate?
• Is the bug vs feature vs question categorization correct?
• Does the issue have a concise meaningful subject?
• are details such as the product version, data to reproduce, etc included
• (re)assigns component/category tags

A volunteer with a good understanding of the issue tracker and tag set should be able to triage a new issue in just a few minutes. It doesn't necessarily need anything heavyweight like attempting to reproduce the problem.

The goal is to provide a safety net/support mechanism where every issue gets at least looked at within N days of being submitted (for some agreed value of N).

Usually issues created by project members would be considered already triaged, so they'd automatically remove the To Be Reviewed tag when creating them. Note also that Github has a bulk edit facility, so if the 24 Wikibase issues with this tag, mostly created by project members, shouldn't have the tag, it can be removed in one swell foop. I just used this facility to remove the To Be Reviewed tag from 99 closed issues on the assumption that they've been appropriately reviewed. 😁

You can see the mechanism working as intended here where the To Be Reviewed tag was removed and the Duplicate tag added when the issue was closed, but the issue #4865 that it points to has no tags other than Bug & To Be Reviewed, so still needed to be triaged, so I added the Mac & Maven tags and removed the To Be Reviewed tag.

I think there's value in having a well-groomed backlog and that the triage process is a net value-add, but it's really up to the project team to figure out what works best for them.

Tom

[Antonin Delpeuch (lists)]

Hi Tom,

Thanks for that and your triage! I agree with you on all points.

On 06/10/2022 16:57, Tom Morris wrote:
> Note also that Github has a bulk edit facility, so if the
> 24 Wikibase issues with this tag

> <https://github.com/OpenRefine/OpenRefine/issues?q=is%3Aissue+is%3Aopen+label%3A%22to+be+reviewed%22+label%3Awikibase>, mostly created by project members, shouldn't have the tag, it can be removed in one swell foop. I just used this facility to remove the To Be Reviewed tag from 99 closed issues on the assumption that they've been appropriately reviewed. 😁

Amazing! How do you use this bulk edit facility? I never discovered it
in the web UI - it does feel like something I would use often!

> I think there's value in having a well-groomed backlog and that the
> triage process is a net value-add, but it's really up to the project
> team to figure out what works best for them.

Yes - so instead of deleting the tag we could we could encourage more
people to chime in the triage, and I wonder how.

I think our tagging scheme is not so great and would be worth making a
bit more consistent, or at least better documented. Perhaps that would
make people more confident that they can make this call.

Antonin

[Tom Morris]

On Thu, Oct 6, 2022 at 12:11 PM Antonin Delpeuch (lists)
<li...@antonin.delpeuch.eu> wrote:
>
> On 06/10/2022 16:57, Tom Morris wrote:

> > Note also that Github has a bulk edit facility, ...

>
> Amazing! How do you use this bulk edit facility? I never discovered it
> in the web UI - it does feel like something I would use often!

In the issue list view, any edits made using the header pulldowns act
on all selected issues. Clicking on the checkbox in the header will
select all issues on the current page. I'm not sure if there's a way
to do more than a page's worth. I ended up doing the 99 issues in four
separate chunks of 25.

> > I think there's value in having a well-groomed backlog and that the
> > triage process is a net value-add, but it's really up to the project
> > team to figure out what works best for them.
>
> Yes - so instead of deleting the tag we could we could encourage more
> people to chime in the triage, and I wonder how.

Two possible suggestions are to 1) ask team members to self-triage for
issues that they create (I just cleaned up all mine) and 2) ask for a
volunteer from the community to take point as the designated triager.
This is probably only an hour or two a week worth of work at the
current bug volumes. A prerequisite for this would be a description of
the desired triage process and/or post-triage state for issues.

For the historical backlog, here are some links for team members to
review their own issues:

https://github.com/OpenRefine/OpenRefine/issues?q=is%3Aopen+is%3Aissue++label%3A%22to+be+reviewed%22+author%3Awetneb
https://github.com/OpenRefine/OpenRefine/issues?q=is%3Aopen+is%3Aissue++label%3A%22to+be+reviewed%22+author%3Atrnstlntk
https://github.com/OpenRefine/OpenRefine/issues?q=is%3Aopen+is%3Aissue+label%3A%22to+be+reviewed%22+author%3Athadguidry
https://github.com/OpenRefine/OpenRefine/issues?q=is%3Aopen+is%3Aissue+label%3A%22to+be+reviewed%22+author%3Aallanaaa
...

> I think our tagging scheme is not so great and would be worth making a
> bit more consistent, or at least better documented. Perhaps that would
> make people more confident that they can make this call.

I agree. One thing worth considering would be whether adding more
prefixes like the current "Priority: " prefix to group like tags
together would be helpful. That would, for example, help all the
component/category tags collate together. Removing/combining obsolete
and/or low frequency usage tags could help make the tag set smaller
and easier to understand.

Tom

[Thad Guidry]

The "too be reviewed" tag also typically sits around when we have a question or an active troubleshooting session with the user that created the issue and we haven't determined what is going on - a real bug, an enhancement needed, etc.

Maybe we should have a "troubleshooting" tag which means, "actively troubleshooting and we don't know yet", but that seems close enough to "too be reviewed" or something like that.

So I'm on the fence on how to categorize or keep initial tags as-is until we know or discover more in the issue with the author like as in https://github.com/OpenRefine/OpenRefine/issues/4899

[Thad Guidry]

I think the "to be reviewed" tag can perhaps stay around after I spent a lot of time going through a lot of our current issues.  I noticed it is helpful to circle back to really verify if 1. a solution was provided for a non-issue, or 2. was properly triaged completely and categorized appropriately.  As Tom stated, I think this is just a matter of us paying more attention and actually dealing with triaging issues well.  I'll continue working on triaging all issues as much as I can over the next few weeks.

[Antonin Delpeuch (lists)]

Thank you everyone for this bug triaging spree! It's really nice to see.
Let's keep it going!

I agree, let's keep the "to be reviewed" tag, the situation is much
different now.

Thank you Tom for this tip about mass actions, I cannot believe I
ignored this bit of GitHub UI for so long!

Antonin

> <https://www.linkedin.com/in/thadguidry/>
> https://calendly.com/thadguidry/ <https://calendly.com/thadguidry/>

>
> --
> You received this message because you are subscribed to the Google
> Groups "OpenRefine Development" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to openrefine-de...@googlegroups.com

> <mailto:openrefine-de...@googlegroups.com>.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/openrefine-dev/CAChbWaOV_%3D0-LYb6jZ_xYoKubFB44pyvPvwP09uMNqRFBHp3rA%40mail.gmail.com <https://groups.google.com/d/msgid/openrefine-dev/CAChbWaOV_%3D0-LYb6jZ_xYoKubFB44pyvPvwP09uMNqRFBHp3rA%40mail.gmail.com?utm_medium=email&utm_source=footer>.

[Tom Morris]

One possibility for making the definition/usage a little crisper would
be to split "To Be Reviewed." OpenLibrary uses "Needs: Triage" and
"Needs: Submitter Input" for the two cases that "To Be Reviewed" is
commonly used for here. (They also have a half dozen other "Needs:
truc/machin" tags, but that seems like serious overkill.)

https://github.com/internetarchive/openlibrary/labels?q=Needs

Tom

[ow...@ostephens.com]

That breakdown (triage vs submitter input) definitely sounds more useful to me in terms of helping us know what needs attention and from whom

Owen

[Thad Guidry]

Let’s change the label “to be reviewed “ to “needs triage “. Then we just need to add a new label for “needs submitter input”.

--

You received this message because you are subscribed to the Google Groups "OpenRefine Development" group.

To unsubscribe from this group and stop receiving emails from it, send an email to openrefine-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openrefine-dev/8bc6a5d2-4b38-4d83-82b8-e089e015d311n%40googlegroups.com.

--

[Antonin Delpeuch (lists)]

Makes sense. I have sometimes used the "not reproducible" tag as "needs submitter input", I don't know if we want to merge those or keep a distinction.

Antonin

To view this discussion on the web visit https://groups.google.com/d/msgid/openrefine-dev/CAChbWaP51Ri2F1Pm1e9LMb8_L%2BPdh_5-jLUfge9j6mRM1X%3Dc-g%40mail.gmail.com.

[Thad Guidry]

I would merge them.  Also we do have the close button that offers in the drop down not reproducible and should use it.  If the OP comes back with a comment to open and gives details to reproduce then we update the tags as necessary to categorize.

I’ll leave it to you to update, manage, and merge the tags.

Then I can get back to triaging all issues.

To view this discussion on the web visit https://groups.google.com/d/msgid/openrefine-dev/0239c02f-8ece-c0c3-49cb-17d673d34451%40antonin.delpeuch.eu.

[Thad Guidry]

I removed the "wont fix" tag (since this is now part of GitHub comment closing (just type a simple explanation and then use dropdown arrow to categorize as won't fix, can't repro, duplicate, stale) I.E. we did no work to resolve). See screenshot below:

I also renamed the "not reproducible" tag to something that more resembled the usage we are expecting with it, which was "waiting on submitter feedback".

Finally a question, which is fairly important to us...

Getting a report on only NEW FEATURES is impossible, but with the above changes, it could be done easily.

Getting a report on IMPROVEMENTS OF EXISTING FEATURES (not legitimate bugs), could be done more easily.

1. Knowing when a new feature is being proposed. "enhancement"  - i.e. enhance the software with a new feature.  This was a default GitHub tag and found throughout the ecosystem, so no change needed I would say for this. (although the English interpretation of it is sometimes actually 2. so we need a much better label for 2.)

2. Knowing when an existing feature needs to be improved. "usability" - i.e. make an existing feature more useful.  A better label name might be "improvement" in keeping with the style of "enhancement".

For reference, here were the default GitHub tags:  Managing labels - GitHub Docs

I thought about adding "Improvement" as a new template option, but that might muddy the waters too much for our users and best if we just triage well and apply the labels (and better labels) consistently.

Thoughts ?

[Thad Guidry]

typo... "but with the below changes"