Engineering Effectiveness Newsletter (May 2022 Edition)

[Dave Lawrence]

Welcome to the May edition of the Engineering Effectiveness Newsletter! The Engineering Effectiveness org makes it easy to develop, test and release Mozilla software at scale. See below for some highlights, then read on for more detailed info!

Highlights

• Welcome to Graham who joins the Engineering Workflow team!

• Engineering Effectiveness is hiring! If you know of anyone who could be interested in working on CI and Automation or on Windows Integration, consider referring them for a bonus!

Detailed Project UpdatesBugzilla and Bugbug

• Suhaib implemented an autonag script to identify and highlight needinfo requests that are pending on inactive users.

• Suhaib implemented autonag scripts to improve the consistency and get traction on urgent web compatibility bugs by suggesting increasing the severity for bugs with P1 WebCompat priority.

• Suhaib implemented an autonag script to suggest increasing the severity of bugs marked as blocking or tracked for nightly, beta, or release.

• Suhaib implemented an autonag script to help get traction on top-crash bugs by suggesting increasing the severity for bugs with topcrash keywords.

• Sebastian started measuring autonag needinfo requests per week, responsiveness by request type and how often the needinfoed person takes the intended type of action

  • This will help us evaluate autonag scripts and figure out which ones need improvements or should be dropped

• Dkl spent quite a bit of a time cleaning up the Bugzilla test suite to help with new contributors.

• Marco improved autonag to avoid sending needinfo requests to other bots.

• Marco adjusted autonag to request Beta uplift right away instead of waiting for a few days after the Nightly landing.

• Suhaib improved the escalation mechanism in autonag to include the management chain when escalating beyond the direct manager.

• Marco upgraded bugbug to use xgboost 1.6.0, for improved performance

Build System and Mach Environment

• Alex Hochheiden added the proposed additional `./mach doctor` check for Artifact Builds

CI and Treeherder

• Bastien and Valentin, with help from Sebastian, modernized the software stack (Celery 5, Django 3.2, Python 3.8) and dropped the unsupported dependency neutrinojs to reduce maintenance and improve performance

• Eva implemented a tool to notify sheriffs when backfills are finished in mozci

• Eva implemented the ability to trigger backfills in mozci

Crash Management

• Gabriele updated dump_syms to ensure we can unwind crashes caused by exceptions thrown in some Microsoft libraries

OS Integration and Security

• We are hiring a Staff or Senior engineer to work on Windows in the OS integration team.

• Utility Process for Audio Decoding is riding the trains for Firefox 102. This strengthens the sandbox for audio decoders by splitting them from video decoders that need more permissions (like hardware access).

• Parts of our refactoring of the Shared Memory APIs have landed. Was blocked on a media decoding crasher, which turned out to be a bug in ffmpeg: http://ffmpeg.org/pipermail/ffmpeg-devel/2022-May/297028.html

• Looking at fallout from win32k Lockdown. Some users with a broken configuration of Exploit Protection have issues, looking at automatic mitigation for the Firefox 102 ESR cycle.

• Dealing Pwn2Own fallout - sandbox escape.

PDF.js

• Thanks to Tom, we had the first automated release of pdf.js in mozilla-central using updatebot: 

  • https://hg.mozilla.org/mozilla-central/rev/66eb7c61e75ad0fc2af4bd866a0caf4b4466a8b7

Phabricator , moz-phab, and Lando

• Sheehan released moz-phab v1.1.0

  • Highlights

    • Fixes for issues around Glean telemetry

    • `moz-phab uplift` command for upcoming uplift approval changes

    • `moz-phab reorg` now preserves revision state

    • Added several warnings for misconfiguration of hg, Python

  • https://wiki.mozilla.org/MozPhab/Changelog#1.1.0

• Zeid released Lando API release-20, which included among other things:

  • major package upgrades

  • total revamp of database migration handling, which makes development faster and easier

  • upgraded to using sentry-sdk, which improves error reporting on production and development servers

Release Engineering and Management

• Release Engineering would like to welcome (back!) Johan Lorenzo

• Gabriel and gbrown have prepared infrastructure for esr-102

• With gbrown's change to authenticode signing, all Firefox Windows installers are now signed with a sha-256 digest

• gbrown helped ship a new Widevine CDM

Version Control

• Sheehan fixed a bug in firefoxtree that caused pulls of esr102 to pull esr10 instead.

mozregression

• Zeid updated Python dependencies generation and handling for different platforms and Python versions, to improve consistency when running tests in CI as well as fix broken CI.

Thanks for reading and see you next month!