Upcoming Firefox NodeJS 8 build requirement (soft for Fx 63, hard for Fx 64)
Dan Mosedale
In early July, we added --enable-nodejs as an optional feature to configure. This was a first step towards being able to use Node-based JavaScript and CSS tooling (e.g. Babel, PostCSS, bundlers like Webpack, etc.) as part of the regular build process. We plan to set --enable-nodejs as the default on August 15th, thereby requiring Node to build Firefox by default. It will be possible to disable this flag for several weeks. After the transition period, there will be a hard requirement on Node to build Firefox.
How will this affect me?
Developers who don’t have Node 8.11.0 or later installed will need to get it:
Execute `mach bootstrap --no-system-changes`, and a Mozilla-private copy of NodeJS will be installed in your home directory (~/.mozbuild/node). configure will always prefer that copy of Node to anything in your $PATH.
This should just work for Mac, Windows, and key versions of Linux (currently expected to be _at_least_ Ubuntu LTS 16.04, Debian 9.5 Stable/Stretch, and CentOS 7), including for Android targets. See below for more details on other versions, OSes, and distributions.
If this causes a problem and you don’t have time to debug it now, just add `ac_add_options --disable-nodejs` to your .mozconfig, which will buy ~4 weeks of time to debug this issue. Please DO file a bug in Firefox Build System :: Bootstrap Configuration, and CC dm...@mozilla.org so that we can look into the problem!
The version of Linux that I develop on wasn’t listed above! What should I do?
Chances are reasonable that your version of Linux will just work. This particularly applies to newer versions of the distributions above, but likely applies to some other distributions as well.
Once `mach bootstrap --no-system-changes` (bug 1481693) lands in the tree, people can start testing and updating our compatibility spreadsheet. I’ll post to dev-builds, dev-platform, and firefox-dev when that happens.
If your OS version hasn’t been tested, please try running `mach bootstrap --no-system-changes`, and if there’s a problem, please update the spreadsheet and file a bug in Firefox Build System :: Bootstrap Configuration, and CC dm...@mozilla.org. Patches, of course, are gladly accepted.
If we made a mistake, and there’s a version that you think needs to be in the “must works” list above, please file a bug in Firefox Build System :: Bootstrap Configuration, and CC dm...@mozilla.org
Maintainers of Firefox builds for OSes which ship Firefox
You’ll need to provide access to NodeJS 8.11.0 or newer in your $PATH when building Firefox.
You can manage the installation any way you like (native package managers seem like likely choices, and https://nodejs.org/en/download/ may prove useful)
We will accept patches for `mach bootstrap` if you wish to make this method more broadly available. It may be worth basing patches on those already in bug 1470332.
What does the current schedule look like?
Thursday, August 9 (mc=63): `mach bootstrap --no-system-changes` can now install a Mozilla-private NodeJS toolchain in ~/.mozbuild (and developers/distros can start testing the bootstrap support)
Weds, August 15 (mc=63, 8 days before soft freeze): --enable-nodejs becomes the default for mozilla-central (distros or developers must install node; those who have issues can set --disable-nodejs, which will be valid until the early part of the 64 nightly cycle)
if high-risk/hard-to-fix problems found, we would simply revert this change for 63
Make GO/NO-GO decision re riding trains
after discussion with relman, this seems not worthwhile, since problems would most likely happen at or immediately after landing time.
Tues, September 11 (mc=64): nodejs becomes a hard dependency for Firefox builds when --disable-nodejs is removed.
For those who want more detail about each of these steps, this work is being driven as part of an Activity-Stream project with a more in-depth roadmap
Can I set up and check my machine/distro beforehand to avoid problems once Node becomes a hard requirement?
Yes, PLEASE! When we land the `mach bootstrap --no-system-changes` support for Node, we’ll post more detailed instructions on how to debug issues ahead of time.
What about node_modules and npm/yarn packages?
No new packages from the npm or yarn registries are landing as part of this patch.
In the immediate future, individual packages will be vendored in on a case-by-case basis.
Immediately after NodeJS lands, and before we start landing packages into the top-level node_modules, I’ll be drafting some straw-man policies for packages and their dependencies (i.e., specific details on what sorts of packages will be allowed, how we’ll handle security, licenses, etc). I’ll be posting to the usual places (mozilla.dev.builds, mozilla.dev.platform, firefox-dev) requesting comment and further discussion.
What’s next for Node in the tree?
The first customer for this code is expected to be DevTools, who will use a standalone version of Babel to transpile JSX to JS files as part of the build (bug 1461714), so that checking in JS build artifacts is no longer necessary. This will be a first step in making risk-analysis / bug-finding / uplifting more efficient for release engineers, sheriffs, and others.
Drafting, circulating, and iterating on the node_modules proposal described above.
Landing an initial prototype of a single node_module with dependencies (possibly PostCSS)
Where can I read a more detailed analysis as to why we’re doing this?
The detailed rationale, along with important tradeoffs and risks, are described in Nick Alexander’s Intent-to-require document, which was posted for comment to dev-platform and dev-builds in February. Note that a few parts of the document are out of date, and I’ve attempted to annotate those parts as comments in the document.
There is still a problem/concern/issue that I think needs to be addressed before the landing happens. What should I do?
For technical/process questions, feel free to respond to the mailing list where you saw this question, email me directly, or ask in #go-node in slack.
For bigger issues (eg “I don’t think this should land yet”, “there’s a big problem that’s likely to affect one or more groups of people”), feel free to reach out to one of: Dan Mosedale (me) <https://mozillians.org/en-US/u/dmose/>, Greg Szorc (build module owner - https://mozillians.org/en-US/u/gps/), or Kim Moir (engineering manager, build and vcs - https://mozillians.org/en-US/u/kmoir/).
Thanks muchly to a bunch of folks who have helped make this happen; most especially Nick Alexander and Greg Szorc, but also (in alphabetical order) Alexander Poirot, Chris Karlof, Jason Laster, Kate Hudson, Kim Moir, Mike Shal, and Tim Spurway. Apologies to anyone I’ve inadvertently forgotten!
Dan
dmos...@mozilla.com
>
> [message HTML auto-conversion elided]
>
Wow, the mailing list forcibly autoconverted that message in a fairly painful -- sorry about that; in the future, I'll stick to plain text here. For anyone who wishes to read something more legible,
https://docs.google.com/document/d/1SsFx5FDlY5JsM421opRM8nuWaTLMiqNZajrhsIL8jow/edit?usp=sharing
Dan