Engineering Effectiveness Newsletter (April 2022 Edition)
Marco Castelluccio
Welcome to the April edition of the Engineering Effectiveness Newsletter! The Engineering Effectiveness org makes it easy to develop, test and release Mozilla software at scale. See below for some highlights, then read on for more detailed info!
Highlights
Simplified tracking of intermittent bugs beta period is complete and we didn’t encounter any hiccups. We will start rolling it out more rapidly in May.
Win32k Lockdown is running as an experiment on release and will be gradually rolled out during the Firefox 100 cycle (for Windows 10 Fall Creators Update & later).
In parallel, the Windows Spotlight work has kicked off and we started landing quality improvements and bug fixes for long-standing annoyances.
pdf.js is now using OffscreenCanvas for improved performance and memory use.
We now have Glean probes reporting CPU use. These new sources of data will help prioritize efforts to reduce Firefox power consumption.
On all channels we have a breakdown of the CPU time used per process type, and of thread wakeups per process type.
On Nightly we additionally have a breakdown of the CPU time per thread name and thread wakeups per thread name.
Engineering Effectiveness is hiring! If you know of anyone who could be interested in working on CI and Automation or on Windows Integration, consider referring them for a bonus!
Contributors
Devon Burriss (dburriss)
manisha27
Detailed Project Updates
Bugzilla and Bugbug
Marco and Suhaib implemented an improvement to autonag that enables prioritizing and limiting the number of actions created by autonag to reduce noise.
Suhaib implemented an autonag script to automatically confirm bugs that have an affected Firefox version flag.
Suhaib implemented an autonag script to confirm bugs with crash signatures.
Suhaib implemented an autonag script to ask follow-up patches to be attached to newbugs by commenting on bugs that receive patches after being closed.
Build System and Mach Environment
Mitch removed the Python 3.6 + Linux restriction from “./mach vendor python”: any environment can now vendor Python code.
Mitch untangled Mach usage in CI to no longer use the system-installed Python packages by default. This enables updating packages such as pylint/pytest/etc without encountering conflicts in esoteric CI tasks.
Mitch added integration tests to ensure that virtualenv activation remains consistent between the main “./mach <command>” process and subprocesses.
Builds were switched to clang 14 and rustc 1.60.
Windows builds now have most of the same warnings enabled as builds for other platforms.
CI and Treeherder
Fission by default is almost 100% complete, all tasks are now either [-fis, -nofis, 1proc].
New Android phones for performance are live, s/Moto G5/Samsung A51/.
Simplified tracking of intermittent bugs beta period is complete, will start rolling out more rapidly in May.
ahal migrated Taskgraph to GitHub, and also enabled code coverage.
Eva wrote a script to compare different mozci algorithms for autoclassification.
Crash Management
Gabriele improved the reliability of crash generation by fixing a long-standing race and deadlock.
Ray further improved the reliability of crash generation by reducing the chance of hitting stack overflows when generating crash reports on Windows.
Gabriele added a crash annotation that describes how the user submitted a crash.
Gabriele imported Sentry changes to dump_syms that will improve stack unwinding in 32-bit x86 crashes.
Fuzzing and Sanitizers
Tyson and Jesse are scaling up fuzzing on Android. Fuzzers are targeting geckoview_example via Grizzly.
Automatic platform specific test case reduction via Grizzly is up and running.
Implemented a TC hook for monitoring fuzzing prefs that are blocked by open bugs.
Reduced WebGL value generation failures by >50% which have a significant impact on coverage depth.
In an effort to support Thunderbird’s security and stability, we have helped them to get something like the ASan Nightly Project to work with Thunderbird.
Lint, Static Analysis and Code Coverage
Devon Burriss (dburriss) implemented additional Java metrics in rust-code-analysis
OS Integration and Security
We shipped X11 connection removal to release in Firefox 99, closing a sandboxing escape on older distros.
Win32k Lockdown is running as an experiment on release and will be gradually rolled out during the Firefox 100 cycle (for Windows 10 Fall Creators Update & later).
The Windows Spotlight work has kicked off and we started landing quality improvements and bugfixes for long-standing annoyances.
Utility Process work has progressed to a point where audio decoding can be enabled by flipping the prefs in Nightly: media.utility-process.enabled.
We now hide your cursor on Windows when you start typing. This can be toggled with widget.windows.hide_cursor_when_typing.
PDF.js
Calixte implemented usage of OffscreenCanvas in the pdf.js main thread and in the worker
It helps drawing some images in the worker instead of the main thread
Overall improving performance and memory use
Calixte added a cache for mask images. The pdf from bug 857031 is drawing way faster now.
Power use
We now have Glean probes reporting CPU use. These new sources of data will help prioritize efforts to reduce Firefox power consumption.
On all channels we have a breakdown of the CPU time used per process type, and of thread wakeups per process type.
On Nightly we additionally have a breakdown of the CPU time per thread name and thread wakeups per thread name.
“Fun” fact: while working on this new data, it became apparent that some process type labels were transmitted incorrectly, with only 1 bit of difference with correct labels. This inspired a This Week in Glean blog post: What Flips Your Bit?.
Release Engineering and Management
Gabriel added debsign support to the taskcluster signing worker. Signing tasks can now sign debian packages using autograph.
Other
manisha27 has improved the Firefox Contributors’ Quick Reference to leverage the bootstrap script.
Thanks for reading and see you next month!