Triage of layout bugs

[L. David Baron]

Right now there are a lot of poorly triaged layout bugs lying around,
and it's somewhat hard to get a good idea of what bugs we have and how
frequently they occur. I'm thinking of trying to write some sort of a
guide to triaging layout bugs. But first, some random thoughts on how I
think the process ought to work...

I'd like to propose a general definition of what should be required to
confirm a bug, and then say what that definition means for layout bugs.
My proposed general definition (with some numbers that I'm basically
making up, but which give an idea of what I mean) is:

* it should be possible to tell what behavior would need to change for
the bug to be considered fixed

* the summary of the bug should accurately and consisely (within the
text input) describe this behavior, but may contain additional
information to search on (e.g., stack signatures, popular URLs)
outside of the size of the text input

* the person confirming the bug should be at least 90% sure that the bug
is valid (note that WONTFIX bugs are valid)

* the person confirming the bug should be at least 50% sure that the
bug is not a duplicate

What does this mean for Layout bugs? It's usually hard to demonstrate
validity without a simple testcase. (It's often easy to demonstrate
invalidity without one, though.) This means to be 90% sure that the bug
is valid, it needs a simplified testcase, and the person confirming the
bug needs to know enough about the relevant specifications to know what
the correct behavior of that testcase should be. Furthermore, it's just
as hard to know if something is a duplicate without a testcase. I think
many of the existing layout bugs are duplicates of a relatively small
number of issues (some of which are invalid), but it's hard to tell
without testcases. (An exception to the need for a simple testcase
might be bugs where a date of regression is known accurately. Recent
regressions should reach developers quickly even if it's hard to make a
testcase, while the changes that caused the regression are fresh.)

It's also worth noting that, in layout, invalid bugs can be valid if
they cause significant problems on major sites, since we can add a
quirk, but the summary of such bugs should probably begin with [quirks]
to indicate that it covers our behavior in quirks mode.

I think there are currently people who confirm layout bugs without
knowing if they are valid. This situation may be helped by moving the
areas of the core (everything in the GRE, basically) into another
product rather than "Browser", which would make sense anyway, and will
hopefully happen soon. Having a clear distinction might make it easier
to do things such as having an IRC channel for discussion of triaging
bugs in the core.

-David

--
L. David Baron <URL: http://dbaron.org/ >

[Boris Zbarsky]

L. David Baron wrote:

> (An exception to the need for a simple testcase
> might be bugs where a date of regression is known accurately. Recent
> regressions should reach developers quickly even if it's hard to make a
> testcase, while the changes that caused the regression are fresh.)

Another exception may be performance bugs, where the complexity (or at
least size) of the testcase is what's exercising whatever O(N^2)
hang-like behavior we happen to have in that bug. ;)

> I think there are currently people who confirm layout bugs without
> knowing if they are valid.

We should make it clear that "I see that too" is NOT grounds to confirm
a bug...

-Boris