Searchfox Update: Searchfox's feud with JS modules is over!

[Andrew Sutherland]

You might have noticed that Searchfox hasn't been bringing its "A game"
to `.mjs` files.  In fact, it might seem like as soon as you used
"import" or "export" in a JS file that searchfox immediately refused to
semantically index the file!  This is because that is what was happening.

This has now been addressed[1] and a quality ratchet has been installed
so that any new parse failure modes for JS will be reported as part of
searchfox's warning emails[2].

Note that Searchfox's feud with JSX is currently ongoing[3], so JSX
files will continue to be unhappy.  But there is hope on the horizon...

The "whenever someone has a good bit of free time to spend on searchfox,
please please please talk to your manager" plan for JS support in
searchfox is tracked at
https://bugzilla.mozilla.org/show_bug.cgi?id=1740290 but is basically to
use https://github.com/sourcegraph/scip-typescript to run typescript
analysis over everything.  This will let us move from a soup symbol
model where we assume every JS file in the tree is loaded into a single
global to instead being able to know everything VS code knows about your
JS code plus searchfox's extra linkage magic like understanding
XPCOM/XPConnect stuff.  Emilio is currently laying the groundwork for
this by moving our rust analysis from rustc save-analysist to
rust-analyzer's new SCIP output[4].

If you're as excited about the future of searchfox as I am, keep an eye
out for my blog posts on a vision for searchfox and a roadmap for
searchfox that should show up on https://planet.mozilla.org/ in the near
future.

Andrew


1: https://bugzilla.mozilla.org/show_bug.cgi?id=1789329 implemented
explicit module parsing for `.mjs` files and fallback module parsing for
JS files.  In https://bugzilla.mozilla.org/show_bug.cgi?id=1792322
(covered more in the next footnote) I also discovered a feud involving
private class symbols due to searchfox's JS analyzer interpreting
everything that starts with `#` as a preprocessor directive. Whoops!

2: https://bugzilla.mozilla.org/show_bug.cgi?id=1792322 improved warning
reporting and landed a ton of interventions to JS parsing so that we
don't report known limitations or suspected intentional parse problems
like files that intentionally have syntax errors. This keeps our
baseline at 0 expected warnings so that when a new warning shows up we
will see it.  Searchfox is always looking for contributors to help with
things like this, so if you would like to be on the email list that will
keep you in the know about when indexers fall over or experience
warnings, please give a shout out in
https://chat.mozilla.org/#/room/#searchfox:mozilla.org please.

3: We currently use a SpiderMonkey JS shell which for obvious reasons
does not support JSX.  There's good potential for a joke about E4X here,
though!

4: https://bugzilla.mozilla.org/show_bug.cgi?id=1761287 tracks the bug
to move from save-analysis to rust-analyzer SCIP.