Engineering Effectiveness Newsletter (Summer 2023 Edition)
Andrew Halberstadt
Welcome to the Summer 2023 edition of the Engineering Effectiveness Newsletter! The Engineering Effectiveness org makes it easy to develop, test and release Mozilla software at scale. See below for some highlights, then read on for more detailed info!
Highlights
Firefox Translations has been released to 50% of the population with Firefox 117! It will be fully released in Firefox 118, with an experiment to tune the behavior of the translation popup. There is a lot of positive press coverage!
Mach commands are now loaded on demand, resulting in a much snappier mach experience! Additionally Mach’s site dependencies were cleaned up, which will result in much fewer Python package conflicts for command authors.
PDF.js now has the ability to add or edit images in PDF files!
Test manifests have begun migrating to TOML! This will make it possible to use automated tooling to modify them as well as switch us to a well known format.
The effort to better support Wayland is well underway, with several test suites now running with Wayland on trunk branches!
Contributors
Jonas Jenwald
Detailed Project Updates
Bugzilla and Bugbug
Dave Lawrence added a What Should I Work On Next? report to Bugzilla to help individual engineers prioritize their work.
Suhaib Mujahid adjusted BugBot to avoid closing inactive bugs whose their status was confirmed.
Suhaib Mujahid expanded BugBot's supported product to include the Release Engineering product.
Build System and Mach Environment
Alex Hochheiden refactored how Mach commands are loaded and when virtualenvs are activated, resulting in most commands initializing faster.
Alex Hochheiden reduced the minimal set of Python dependencies in the mach site, resulting in less restrictive command sites (eg: build, lint, docs, etc). This makes it easier to resolve dependency conflicts in existing sites (except common) and makes changing a command’s site from common to a new site much more effective than before.
Serge Guelton updated vendoring of several third party sources through mach vendor
Serge Guelton enabled full+cross LTO by default for shippable Linux and Windows build
Serge Guelton removed duplicate sources for double-conversion
Serge Guelton increased the number of Taskcluster tasks run on Taskcluster-built Python.
CI and Treeherder
Andrew Halberstadt removed the multi_dep loader from gecko_taskgraph and replaced it with Taskgraph’s new from_deps transforms.
Andrew Halberstadt stood up several test suites running with Wayland, with progress made on several others.
Tom Marble migrated two batches of .ini manifests to the new .toml format (see newsgroup post for details).
Teklia contractors switched the generated Treeherder API documentation from CoreAPI to maintained OpenAPI
Crash Management
Gabriele Svelto switched minidump generation for child process crashes on Linux/Android using ARM/AArch64 processors to use the oxidized minidump writer.
Gabriele Svelto ensured process crashes print their PID and type in test automation.
Gabriele Svelto started automatically scraping symbols for Windows graphics drivers from Intel, Nvidia and AMD.
Suhaib Mujahid implemented some improvements to the automated bug filing of new actionable crashes:
Include more relevant details in the bug descriptions.
Expand the analysis timeframe for generating bug descriptions
Fill regression-related metadata (e.g., release status flags)
Improve the detection of use after free crashes (thanks to Dan Veditz and Gabriele Svelto)
Lint, Static Analysis and Code Coverage
Serge Guelton added a linter for third-party license paths
OS Integration and Security
Bob Owen shipped new process sandbox mitigations in Fx117
MITIGATION_IMAGE_LOAD_NO_REMOTE & MITIGATION_IMAGE_LOAD_NO_LOW_LABEL flags for all processes when not running from remote drive
MITIGATION_NONSYSTEM_FONT_DISABLE enabled for RDD, Utility, GMP and Socket processes
JOB_OBJECT_LIMIT_ACTIVE_PROCESS set to 1 for GPU process
CET compatible modules only for the GMP and GPU processes
Greg Stoll allowed multiple DLL blocklist flags to more tightly target blocks (bug 1835926)
Greg Stoll made some Microsoft published DLLs (which should not be blocked!) not show up on about:third-party (bug 1817026)
PDF.js
Calixte Denizet implemented the ability to add images in a PDF
Calixte Denizet added more keyboard shortcuts in order to be able to edit.
Calixte Denizet improved the way to resize editors.
Calixte Denizet added the ability to print filled forms in Firefox for Android.
Jonas Jenwald fixed some bugs, improved the overall quality of the code and reviewed a lot of patches.
Firefox Translations
Firefox Translations has been released to 50% of the population with Firefox 117! It will be fully released in Firefox 118, with an experiment to tune the behavior of the translation popup.
There was already some positive press coverage:
Serge Guelton improved translation speed on aarch64 through an update of gemmology.
Gabriel Bustamante enhanced Taskcluster to support the long-running tasks in the Firefox Translations training pipeline.
Work to support translations on Android has started. Follow along on the meta bug!
Power use
In a nice blog post, the Green Web Foundation explains how the Firefox Profiler became the first major browser developer tool to display carbon emissions alongside other performance data, using the Green Web Foundation’s CO2.js library.
Phabricator , moz-phab, and Lando
Connor Sheehan deployed an API endpoint to Lando that accepts patches for submission to try. Once mach try is migrated to use the new endpoint, submission times will take seconds in the worst case instead of minutes.
Connor Sheehan made Lando strip “Depends on” text from commit messages, which prevents unnecessary linking of uplift Phabricator revisions.
Zeid Zabaneh implemented a feature in Lando that will record the Bugzilla IDs of users who approved a revision at landing time, and of those users, identifies who are owners or peers. The latter will only take place on repos that implement a mots.yaml file.
Release Engineering and Management
Geoff Brown documented the steps to de-suport an OS on Firefox Desktop. This followed the migration of Windows 7/8 and the macOS <10.14 users to ESR.
Heitor Neiva and the Pocket team moved the Pocket apps on the Apple App Store under Mozilla's account. Until then, Pocket had their own account.
Gabriel Bustamante made a visualization of several Taskcluster graphs.
Julien Cristau centralized release-related notifications for the Mozilla VPN to #releaseduty (Element).
Andrew Halberstadt coordinated the migration of Firefox CI artifacts to GCS
Andrew Halberstadt integrated Application Services with Ship-It, leveraging the new mozilla-taskgraph project to do so in a reusable way.
Ben Hearsum added a new feature in taskgraph: the ability to substitute any values without the need for creating your own specific transform.
The 2024 Firefox Release Calendar has been finalized,
https://whattrainisitnow.com/ has been updated, please see the calendar for an overview or the monthly calendar for a detailed overview.
Mozregression
Zeid Zabaneh added support for aarch64 builds for geckoview-example.
Zeid Zabaneh and Geoff Brown identified and updated some mozbase packages (mozdevice & mozprocess) that were causing bugs in mozregression on Windows.
Zeid Zabaneh added failsafe functionality for missing taskcluster artifacts to prevent fatal errors.
Zeid Zabaneh and Heitor Neiva fixed up signing and notarization on macOS, enabling simple upgrades on macOS without the need to disable gatekeeper.
Zeid Zabaneh added a splash screen that fixes a usability issue on macOS that was caused by the new changes.
Zeid Zabaneh also added support for signing the Windows installer of mozregression
As of version 5.9.0, binaries for macOS and Windows will be signed (and notarized, in the case of macOS). Note that there is a delay between when the GitHub release is published, and when the signed and notarized binaries are uploaded for each new release.
Other
Kagami Rosylight added review groups support to mots
Serge Guelton avoided thread lock on some TLS access
Serge Guelton removed some global constructor calls
Thanks for reading and see you next month!