Rust and --enable-shared-js

[Henri Sivonen]

There's an effort to add Rust code to SpiderMonkey:
https://bugzilla.mozilla.org/show_bug.cgi?id=1490948

This will introduce a jsrust_shared crate that will just depend on all
the Rust crates that SpiderMonkey needs like gkrust_shared depends on
the crates the rest of Gecko needs.

This is fine both for building standalone SpiderMonkey (a top-level
jsrust will produce a .a and depend on jsrust_shared) and SpiderMonkey
as part of libxul (gkrust_shared will depend on jsrust_shared).

However, there exists a third configuration: --enable-shared-js. With
this option, SpiderMonkey is linked dynamically instead of being baked
into libxul. This is fine as long the set of FFI-exposing crates that
SpiderMonkey depends on and the set of FFI-exposing crates that the
rest of Gecko depends on are disjoint. If they aren't disjoint, a
symbol conflict is expected.

AFAICT, this could be solved in at least three ways:

1) Keeping the sets disjoint. If both SpiderMonkey and the rest of
Gecko want to call the same Rust code, introduce a differently-named
FFI binding for SpiderMonkey.

2) Making FFI symbols .so-internal so that they don't conflict
between shared libraries. Per
https://bugzilla.mozilla.org/show_bug.cgi?id=1490603 , it seems that
this would require rustc changes that don't exist yet.

3) Dropping support for --enable-shared-js

For my immediate use case, I want to make 4 functions available both
to SpiderMonkey and the rest of Gecko, so option #1 is feasible, but
it won't scale. Maybe #2 becomes feasible before scaling up #1 becomes
a problem.

But still, I'm curious:

How important is --enable-shared-js? I gather its use case is making
builds faster for SpiderMonkey developers. Is that the only use case?
Is it being used that way in practice?

--
Henri Sivonen
hsiv...@mozilla.com

[Mike Hommey]

for _Gecko_ developers.

Mike

[Boris Zbarsky]

On 9/24/18 4:04 AM, Henri Sivonen wrote:
> How important is --enable-shared-js? I gather its use case is making
> builds faster for SpiderMonkey developers.

My use case for it is to be able to use the "exclude samples from
library X" or "collapse library X" tools in profilers (like Instruments)
to more easily break down profiles into "page JS" and "Gecko things".

That said, I haven't done that very much recently...

-Boris

[Henri Sivonen]

On Mon, Sep 24, 2018 at 3:24 PM, Boris Zbarsky <bzba...@mit.edu> wrote:
> On 9/24/18 4:04 AM, Henri Sivonen wrote:
>>
>> How important is --enable-shared-js? I gather its use case is making
>> builds faster for SpiderMonkey developers.
>
>
> My use case for it is to be able to use the "exclude samples from library X"
> or "collapse library X" tools in profilers (like Instruments) to more easily
> break down profiles into "page JS" and "Gecko things".

OK.

On Mon, Sep 24, 2018 at 1:24 PM, Mike Hommey <m...@glandium.org> wrote:
>> How important is --enable-shared-js? I gather its use case is making

>> builds faster for SpiderMonkey developers. Is that the only use case?
>
> for _Gecko_ developers.

This surprises me. Doesn't the build system take care of not
rebuilding SpiderMonkey if it hasn't been edited? Is this only about
the link time?

What's the conclusion regarding next steps? Should I introduce
js_-prefixed copies of the four Rust FFI functions that I want to make
available to SpiderMonkey?

--
Henri Sivonen
hsiv...@mozilla.com

[Luke Wagner]

(Sorry, I polled #jsapi about this issue back when you first posted and
then forgot to reply with the response.)

It doesn't seem like any SM devs use --enable-shared-js for their own
development but we do know that various embedders (e.g. GNOME) use the JS
shared library and so we'd like to keep that configuration tested and
working. One hybrid option is thus:
- drop support for building Gecko with --enable-shared-js (avoiding the
symbol conflict issue), but keep --enable-shared-js as a configure option
for JS shell builds
- have at least one tier-1 --enable-shared-js JS shell build on automation
so that we at least keep it working

Cheers,
Luke

> _______________________________________________
> dev-platform mailing list
> dev-pl...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
>

[Mike Hommey]

On Tue, Oct 02, 2018 at 03:25:04PM +0300, Henri Sivonen wrote:
> On Mon, Sep 24, 2018 at 3:24 PM, Boris Zbarsky <bzba...@mit.edu> wrote:
> > On 9/24/18 4:04 AM, Henri Sivonen wrote:
> >>
> >> How important is --enable-shared-js? I gather its use case is making
> >> builds faster for SpiderMonkey developers.
> >
> >
> > My use case for it is to be able to use the "exclude samples from library X"
> > or "collapse library X" tools in profilers (like Instruments) to more easily
> > break down profiles into "page JS" and "Gecko things".

I would probably be worthwhile to be able to do that without
--enable-shared-js, only because that's not how Nightlies are shipped,
and this seems like something that can be useful to do on nightlies.
Is grouping by js::* JS* a workable alternative?

> On Mon, Sep 24, 2018 at 1:24 PM, Mike Hommey <m...@glandium.org> wrote:
> >> How important is --enable-shared-js? I gather its use case is making
> >> builds faster for SpiderMonkey developers. Is that the only use case?
> >
> > for _Gecko_ developers.
>
> This surprises me. Doesn't the build system take care of not
> rebuilding SpiderMonkey if it hasn't been edited? Is this only about
> the link time?

Yes, it reduces link time for libxul because libmozjs doesn't need to be
linked statically to it. That might not matter anymore, but it's better
to know if people rely on that.

I'd like to note that making libxul always have libmozjs statically
linked could allow to move some things from libmozglue to libxul.

Mike

[Mike Hommey]

Hi,

I'm revisiting the topic of --enable-shared-js for Gecko builds for
different reasons than the one that started this thread.

On Mon, Sep 24, 2018 at 08:24:43AM -0400, Boris Zbarsky wrote:
> My use case for it is to be able to use the "exclude samples from library X"
> or "collapse library X" tools in profilers (like Instruments) to more easily
> break down profiles into "page JS" and "Gecko things".
>
> That said, I haven't done that very much recently...

And my guess nobody has done that since, nor for a while, because the
way an attempt to build with --enable-shared-js fails leaves me thinking
this has been broken at least since bug 1436179 landed... a year ago.

Considering this has apparently been broken for so long, I guess nobody
will object to me removing the option for Gecko builds?

On Tue, Oct 02, 2018 at 09:29:51AM -0500, Luke Wagner wrote:
> (Sorry, I polled #jsapi about this issue back when you first posted and
> then forgot to reply with the response.)
>
> It doesn't seem like any SM devs use --enable-shared-js for their own
> development but we do know that various embedders (e.g. GNOME) use the JS
> shared library and so we'd like to keep that configuration tested and
> working. One hybrid option is thus:
> - drop support for building Gecko with --enable-shared-js (avoiding the
> symbol conflict issue), but keep --enable-shared-js as a configure option
> for JS shell builds
> - have at least one tier-1 --enable-shared-js JS shell build on automation
> so that we at least keep it working

It's worth noting that --enable-shared-js is already the default when
building spidermonkey standalone.

Mike

[Boris Zbarsky]

On 5/21/19 9:55 PM, Mike Hommey wrote:
> Considering this has apparently been broken for so long, I guess nobody
> will object to me removing the option for Gecko builds?

It's probably fine, yeah...

-Boris

[Mike Hommey]

Now removed on autoland via bug 1554056.

Mike

[Henri Sivonen]

On Tue, May 28, 2019 at 3:16 AM Mike Hommey <m...@glandium.org> wrote:
>
> On Tue, May 21, 2019 at 10:32:20PM -0400, Boris Zbarsky wrote:

> Now removed on autoland via bug 1554056.

Thanks.

It appears that building jsrust_shared is still conditional on
ENABLE_WASM_CRANELIFT. How optional is ENABLE_WASM_CRANELIFT in
practice these days? Is it genuinely optional for Firefox? Is it
genuinely optional for standalone SpiderMonkey? If it is, are we OK
with building without ENABLE_WASM_CRANELIFT having other non-Cranelift
effects on SpiderMonkey performance (e.g. turning off SIMD for some
operations) or on whether a particular string conversion is available
in jsapi.h?

I'm trying to understand the implication of Cranelift being optional
for other Rust code in SpiderMonkey. I'd like to add Rust-backed
SIMD-accelerated Latin1ness checking and UTF-8 validity checking to
SpiderMonkey and Rust-backed conversion from JSString to UTF-8 in
jsapi.h, and my understanding from All Hands was that adding these
things would be OK, since SpiderMonkey already depends on Rust.

--
Henri Sivonen
hsiv...@mozilla.com

[Lars Hansen]

Cranelift should be genuinely optional until further notice; to my
knownledge, no near-term product work in Firefox or SpiderMonkey depends on
Cranelift. Cranelift is present in Nightly but (so far as I can tell) not
in Release. It can be disabled in the JS shell by configuring with
--disable-cranelift, and I just tested that this works. To the extent
there is other Rust code in SpiderMonkey it should not, so far as I know,
depend on the presence of Cranelift. It also seems to me that we should be
able to use Rust in SpiderMonkey independently of whether Cranelift is
there, so if that does not work it ought to be fixed.

--lars

On Wed, Aug 7, 2019 at 3:53 PM Henri Sivonen <hsiv...@mozilla.com> wrote:

> On Tue, May 28, 2019 at 3:16 AM Mike Hommey <m...@glandium.org> wrote:
> >
> > On Tue, May 21, 2019 at 10:32:20PM -0400, Boris Zbarsky wrote:

> > Now removed on autoland via bug 1554056.
>
> Thanks.
>
> It appears that building jsrust_shared is still conditional on
> ENABLE_WASM_CRANELIFT. How optional is ENABLE_WASM_CRANELIFT in
> practice these days? Is it genuinely optional for Firefox? Is it
> genuinely optional for standalone SpiderMonkey? If it is, are we OK
> with building without ENABLE_WASM_CRANELIFT having other non-Cranelift
> effects on SpiderMonkey performance (e.g. turning off SIMD for some
> operations) or on whether a particular string conversion is available
> in jsapi.h?
>
> I'm trying to understand the implication of Cranelift being optional
> for other Rust code in SpiderMonkey. I'd like to add Rust-backed
> SIMD-accelerated Latin1ness checking and UTF-8 validity checking to
> SpiderMonkey and Rust-backed conversion from JSString to UTF-8 in
> jsapi.h, and my understanding from All Hands was that adding these
> things would be OK, since SpiderMonkey already depends on Rust.
>
> --
> Henri Sivonen
> hsiv...@mozilla.com

[Henri Sivonen]

On Tue, Aug 13, 2019 at 2:18 PM Lars Hansen <lha...@mozilla.com> wrote:
> Cranelift should be genuinely optional until further notice; to my knownledge, no near-term product work in Firefox or SpiderMonkey depends on Cranelift. Cranelift is present in Nightly but (so far as I can tell) not in Release. It can be disabled in the JS shell by configuring with --disable-cranelift, and I just tested that this works. To the extent there is other Rust code in SpiderMonkey it should not, so far as I know, depend on the presence of Cranelift. It also seems to me that we should be able to use Rust in SpiderMonkey independently of whether Cranelift is there, so if that does not work it ought to be fixed.

Thanks. That makes sense to me.

The present state (now that
https://bugzilla.mozilla.org/show_bug.cgi?id=1572364 has landed) is
that when built as part of libxul, SpiderMonkey can use Rust code
(jsrust_shared gets built) regardless of whether Cranelift is enabled.
However, when SpiderMonkey is built outside libxul, SpiderMonkey can
use Rust code (jsrust_shared gets built) only if Cranelift is enabled.
I've filed https://bugzilla.mozilla.org/show_bug.cgi?id=1573098 to
change that.

(The actual addition of non-Cranelift Rust code of interest to
jsrust_shared hasn't landed yet.)

--
Henri Sivonen
hsiv...@mozilla.com