Changes to the Firefox Bug Triage Process [ If you use Bugzilla, read this ]

65 views
Skip to first unread message

Emma Humphries

unread,
Apr 27, 2020, 6:26:27 PM4/27/20
to all-mo...@mozilla.com, dev-platform, tool...@lists.mozilla.org
Starting with the next Firefox release cycle, Monday, May 4th, we expect
you to set the Severity field to a non-default value when triaging Firefox
related bugs.

If you use Bugzilla, but not for Firefox, please read on, because there's
changes to the Severity field.

After discussions with engineering management, release management, QA, and
people triaging bugs we’ve decided that we want consistent assessments of a
bug’s severity so that we know what bugs will affect the quality of a
release.

This is a change from what we've been doing historically during triage,
which is setting the Priority field. Using Priority alone conflates
severity with scheduling.

The new definition of Triaged will be Firefox-related bugs of all types
(defect, task, enhancement) where the component is not UNTRIAGED, which has
one or more current (Nightly, Beta, Release, ESR) Status_FirefoxNN flags
set to a non-default value, and a non-default value of Severity.

Bugs of type Task or Enhancement may have a severity of N/A, but defects
must have a severity that is neither ‘--’ or N/A. No bug with a severity of
‘--’ is considered triaged.

We will update our triage dashboards, saved bugzilla queries, and AutoNag
rules to reflect this change.

Please join us on the Bug Handling channel on Riot/Matrix (
https://chat.mozilla.org/#/room/#bug-handling:mozilla.org) with your
questions.

Part of the reason we’ve not consistently used severity is that the field’s
default was NORMAL, so we could not easily determine when it was modified.

By default, a new bug’s severity will be ‘--’ and the severity values will
change to:

--: Default for new bugs

N/A: (not applicable): Only applies to bugs of type Task or Enhancement.

S1: (Catastrophic) Blocks development/testing, may impact more than 25% of
users, causes data loss, potential chemspill, and no workaround available

S2: (Serious) Major Functionality/product severely impaired and a
satisfactory workaround doesn't exist

S3: (Normal) Blocks non-critical functionality and a work around exists

S4: (Small/Trivial) minor significance, cosmetic issues, low or no impact
to users
Use these descriptions to guide your decision on a bug’s severity. We’ll be
mapping existing bugs to the new definitions.

We are making this change so that we can focus on identifying and fixing
critical issues in current and upcoming releases.

You can continue to set Priority when you triage bugs, and it’s suggested
that you do continue to follow the definitions for the values of the
Priority field if you do.

Note that a bug can be a regression but may have a minor severity, and
conversely bugs which are not regressions can have a major severity.

High Severity and tracking are not the same. If in doubt whether a high
severity bug merits tracking, err on the side of requesting it - relman can
turn it down, and we'd rather have more visibility than less. In general,
it is not necessary to request tracking for low-severity bugs.

For bugs triaged by QA, apart from setting needed flags and components, the
severity field will be updated by QA. Triage owners can review and update
severity if needed.

After an S1 defect found in QA or release has been resolved, a Root Cause
should be determined and the RCA program flag for the bug updated. Process
on this topic will be communicated later in a separate email.

If you have scripts, dashboards, or processes that depend on the current
values of the Severity field, please update them. The Bugzilla team will
update your saved searches to reflect the new values of the Severity field.
This new process was the result of review and discussions with: Vicky Chin,
Tania Maity, Ryan VanderMulen, Thomas Elin, Andrew Overholt, Asa Dotzler,
Mike Connoly, Gijs Kruitbosch, and Jared Wein. I'm grateful to them for
their ideas and contributions to this. Any errors are my own.

A longer version of this email with more background can be found at:
https://docs.google.com/document/d/1HYPBDePvYX26JOrk6ymsEeViwTdWhCVL-elSwInsMKw/

In solidarity,

Emma Humphries, Developer Workflow Team
Reply all
Reply to author
Forward
0 new messages