Engineering Effectiveness Newsletter (February 2022 Edition)

[Mitchell Hentges]

Welcome to the February edition of the Engineering Effectiveness Newsletter! The Engineering Effectiveness org makes it easy to develop, test and release Mozilla software at scale. See below for some highlights, then read on for more detailed info!

Highlights

• Major improvements to CI intermittent-failure detection by Marco and Eva! Details are below 👇

• Serious, detailed docs for setting up Taskgraph have been written by ahal, which should significantly improve stability

Detailed Project Updates

Bugzilla and Bugbug

• marco implemented a new autonag script to needinfo authors if their changes cause a regression bug

• calixte and marco implemented a new autonag script to unassign inactive accounts from bugs (example usage here)

• Sylvestre added new autonag checks to escalate bugs with several CCs or votes

Build System and Mach Environment

• Alex fixed a 3 minute hang on Windows when running `./mach python-test` by replacing ProcessHandler with subprocess

• Alex also added a progress bar to `./mach python-test`’s output

• Mitch fixed `./mach lint -l black` on Windows

CI and Treeherder

• arai added a button to task groups which offers a one-click solution to request all tasks in a group to run - commonly used by developers testing their changes on Try to extend the build and tests to run (e.g. empty pushes)

• Eva added support for retrieving automated mozci failure classification results and store them in Treeherder

• marco landed multiple improvements to the mozci intermittent / real failure classification logic (1, 2, 3, 4, 5) 

• Eva implemented a monitoring mode to periodically evaluate mozci results and investigate errors

• Bastien added support for sending emails when mozci detects a push containing a real failure, and Eva for sending Matrix notifications

• Eva built a script to compare mozci classifications with sheriff classifications (1, 2)

• Marco and Eva improved the mozci classification evaluation script to avoid any biases from manual classifications and expanded it (1, 2, 3, 4, 5)

• jmaher got all desktop tests running as fission, and added a Linux “nofis” variant until Android is supported

• jmaher enabled artifact support for browser-chrome a11y tests, split up the large mochitest-media manifests, and “greened up” CI by improving the way some intermittent tests were run

• Chidinma added the ability to retrigger tasks from mozci as well as extended the classify command to auto-retrigger for greater accuracy

Fuzzing and Sanitizers

• Jesse added fuzzing support of Mac OS on Taskcluster

• Jason implemented WebGPU fuzzing in Domino

Lint, Static Analysis and Code Coverage

• marco disabled RLBox in code coverage builds, to make sure we collect coverage data for RLBox-ed code

PDF.js

• Calixte added support for searching for words in PDFs with or without diacritics (e.g. déjà-vu). At the same time, the find-in-page support in PDF has been greatly improved.

• Calixte fixed handling cases where “lineWidth” is too small to render (such as after transforms)

• Brendan resolved a “soft mask”-related issue that could prevent images and text from being rendered properly

• jrmuizel added support for some JPEG2000 COD options

Phabricator , moz-phab, and Lando

• sheehan landed changes to Lando to allow testing against a mock s3 bucket, eliminating the need for a real one

Release Engineering and Management

• Release Engineering would like to welcome Cyrus Bond as their new Engineering Manager!

• Release Engineering would also like to (officially) welcome Michelle Goossens (aka masterwayz) who is starting a contract to help migrate our CI worker pools to GCP

• ahal created comprehensive documentation on getting started with Taskgraph for the first time.

• ahal designed and implemented a way to install Python dependencies from the run-task script. This will allow consumers of Taskgraph to install it from pypi.

• hneiva refactored VPN signing and added support for adhoc signing. This will enable automated signing once builds are moved to Taskcluster.

• Gabriel enhanced the add-ons pipeline to archive and create releases for signed system add-on artifacts.

Version Control

• sheehan added a check for the Python version to the Mercurial setup wizard, to prepare for Mercurial Py2 deprecation

Other

• The NSPR documentation has moved to mozilla-central and is available on https://firefox-source-docs.mozilla.org/nspr/ 

Thanks for reading and see you next month!