PSA: Have bugs automatically filed to you for new failing tests introduced by upstream WPT

[Robert Ma]

Happy new year, Blink developers!

Tl;dr. If you ever write Web Platform Tests (WPT), you can now opt into a new feature of wpt-import to receive automatically filed bugs when new failing tests are brought into Chromium from upstream (GitHub) during an import.

Last year, I sent out an email asking for feedback on import notifications. Based on your inputs, a new feature has been added to wpt-import to enable auto bug filing when new failures are introduced from upstream. Here is the design doc if you are interested in the details.

This is an opt-in feature that can be turned on in individual directories in LayoutTests/external/wpt. When it's turned on, wpt-import will file one bug per import per directory if the directory has new failing tests. The bug will contain a list of new failures and a list of upstream commits (with GitHub links) included in this import. See this section of the design doc for the template of bug reports. Owners listed in the OWNERS file will be CC'ed, and the "# COMPONENT: " line will be used to extract the Monorail component. For now, "new failures" include new TestExpectations lines (usually crashes, timeouts, or failing reftests), and new FAILs in testharness.js-based tests.

How to enroll?

You can easily opt in by adding a new comment line

# WPT-NOTIFY: true

to the OWNERS file of the tests you are interested in (it's also strongly recommended to specify COMPONENT in OWNERS). Here is an example CL that turned on notification for flexbox (many thanks to cbiesinger!). Finally, please send the CL to me (robertma@) so that I can keep track.

Please feel free to reach out to me or ecosystem-infra@ if you have any question.

Best,

Robert