Notes about implementing DOM APIs in Rust

[Henri Sivonen]

Last week, people interested in the matter met to talk about
implementing DOM APIs in Rust. Here are my notes.

The summary of the meeting is that we shouldn't be designing bridge
framework type of stuff ahead of time and for the time being we should
instead address issues as they arise.

We shouldn't expect to be able to use Servo's implementations of DOM
APIs in a drop-in a manner in Gecko. Because Servo allocates Rust
objects on the JavaScript heap, but Gecko doesn't allocate C++ objects
on the JavaScript heap, the memory management arrangements of the two
engines are fundamentally different and, therefore, the same WebIDL
bindings won't work.

Additionally, it was discussed whether there should be Gecko-specific
WebIDL bindings. The conclusion was that there shouldn't be, at least
for now. Instead, when implementing a WebIDL-specified API in Gecko in
Rust, the existing bindings for C++ should be used and then the C++
code should call into Rust.

If it happens that the implementation of an API could be shared
between Servo and Gecko if it weren't for the binding incompatibility,
the implementation of the API should be abstracted away from the
binding so that both Servo-style and Gecko-style binding code can call
into the shared implementation.

As an exception to the general sentiment of not providing WebIDL
bindings for Rust in Gecko, there was agreement that we should
introduce the WebIDL annotation for requesting a USVString to be
exposed as a guaranteed-valid UTF-8 subclass of nsACString instead of
the usual UTF-16. The idea is that it should be easy to pass this
onwards to Rust and view the data as &str in Rust.

Additionally, there was some discussion about reference counting.
Implementing an XPCOM binding for Rust was not a popular idea and was
discarded. Still, it was considered important to be able to use
Gecko-style reference counting in a cycle collector-compatible manner
across the C++/Rust boundary. Specifically, we should introduce a
macro for implementing AddRef, Release and associated cycle collection
methods in Rust in a cycle collector-compatible way and we should
implement a Gecko-flavored replacement for Rc<> that calls Gecko-style
AddRef/Release as appropriate.

Now that I'm looking at the hand-written notes that I made in the
meeting, I notice that the above paragraph fails to say how the
AddRef, Release and associated cycle collection-related calls are
routed from C++ to Rust or from Rust to C++. There was some talk about
vtable hacks, but my recollection is that those got rejected along
with XPCOM. So as far as I can tell, we are left with the meeting not
concluding how exactly these calls across the language boundary.

(A remark of mine that was not discussed in the meeting and that I'm
just adding onto the notes now as hopefully relevant to the last
paragraph: If we don't want to hack vtables to make direct virtual
calls between C++ and Rust, one way to use C linkage and FFI but still
have the C++-side calls syntactically look like C++ method invocations
that fit into existing C++-side smart pointers (etc.) is to interpret
pointers returned from Rust as pointers to instances of a C++ class
whose methods do nothing but call FFI functions with "this" as the
first argument as seen in
https://github.com/hsivonen/encoding-rs/blob/master/include/encoding_rs_cpp.h
.)

--
Henri Sivonen
hsiv...@hsivonen.fi
https://hsivonen.fi/

[Josh Matthews]

On 2016-06-22 1:05 PM, Henri Sivonen wrote:

> Additionally, there was some discussion about reference counting.
> Implementing an XPCOM binding for Rust was not a popular idea and was
> discarded. Still, it was considered important to be able to use
> Gecko-style reference counting in a cycle collector-compatible manner
> across the C++/Rust boundary. Specifically, we should introduce a
> macro for implementing AddRef, Release and associated cycle collection
> methods in Rust in a cycle collector-compatible way and we should
> implement a Gecko-flavored replacement for Rc<> that calls Gecko-style
> AddRef/Release as appropriate.
>
> Now that I'm looking at the hand-written notes that I made in the
> meeting, I notice that the above paragraph fails to say how the
> AddRef, Release and associated cycle collection-related calls are
> routed from C++ to Rust or from Rust to C++. There was some talk about
> vtable hacks, but my recollection is that those got rejected along
> with XPCOM. So as far as I can tell, we are left with the meeting not
> concluding how exactly these calls across the language boundary.

I do not recall dismissing exposing COM-compatible vtables from Rust.

[Jack Moffitt]

> I do not recall dismissing exposing COM-compatible vtables from Rust.

We must implement COM interfaces in Rust for Windows platform things*
already. Specifically I have macros that generate IUnknown which for
Windows COM includes AddRef, Release, and QueryInterface. For COM
interfaces with single inheritance, this isn't too bad. mystor has
also been hacking on similar things

jack.

* Specifically I've been doing it to get DirectWrite working

[Bobby Holley]

Sure - it's just a question of whether this level of hackery is desirable
for integrating with the rest of Gecko (a platform we control). I suspect
that we can probably solve whatever use-cases arise in cleaner ways, but we
should wait for use-cases to appear first.

On Wed, Jun 22, 2016 at 10:33 AM, Jack Moffitt <ja...@metajack.im> wrote:

> > I do not recall dismissing exposing COM-compatible vtables from Rust.
>

> We must implement COM interfaces in Rust for Windows platform things*
> already. Specifically I have macros that generate IUnknown which for
> Windows COM includes AddRef, Release, and QueryInterface. For COM
> interfaces with single inheritance, this isn't too bad. mystor has
> also been hacking on similar things
>
> jack.
>
> * Specifically I've been doing it to get DirectWrite working

> _______________________________________________
> dev-platform mailing list
> dev-pl...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
>

[Andrew McCreight]

On Wed, Jun 22, 2016 at 1:05 PM, Henri Sivonen <hsiv...@hsivonen.fi> wrote:

> Now that I'm looking at the hand-written notes that I made in the
> meeting, I notice that the above paragraph fails to say how the
> AddRef, Release and associated cycle collection-related calls are
> routed from C++ to Rust or from Rust to C++. There was some talk about
> vtable hacks, but my recollection is that those got rejected along
> with XPCOM. So as far as I can tell, we are left with the meeting not
> concluding how exactly these calls across the language boundary.
>

C++ CCed objects don't even all use COM. Maybe the existing non-nsISupports
CC infrastructure could be used for Rust objects. If that doesn't work,
adding explicit support for Rust objects to the CC, like we have for JS,
doesn't seem like it would be too difficult. The CC used to nominally
support arbitrary languages.

Andrew

> (A remark of mine that was not discussed in the meeting and that I'm
> just adding onto the notes now as hopefully relevant to the last
> paragraph: If we don't want to hack vtables to make direct virtual
> calls between C++ and Rust, one way to use C linkage and FFI but still
> have the C++-side calls syntactically look like C++ method invocations
> that fit into existing C++-side smart pointers (etc.) is to interpret
> pointers returned from Rust as pointers to instances of a C++ class
> whose methods do nothing but call FFI functions with "this" as the
> first argument as seen in
>
> https://github.com/hsivonen/encoding-rs/blob/master/include/encoding_rs_cpp.h
> .)
>
> --
> Henri Sivonen
> hsiv...@hsivonen.fi
> https://hsivonen.fi/

[Henri Sivonen]

On Jun 23, 2016 1:33 AM, "Andrew McCreight" <amccr...@mozilla.com> wrote:
>
> On Wed, Jun 22, 2016 at 1:05 PM, Henri Sivonen <hsiv...@hsivonen.fi>
wrote:
>
> > Now that I'm looking at the hand-written notes that I made in the
> > meeting, I notice that the above paragraph fails to say how the
> > AddRef, Release and associated cycle collection-related calls are
> > routed from C++ to Rust or from Rust to C++. There was some talk about
> > vtable hacks, but my recollection is that those got rejected along
> > with XPCOM. So as far as I can tell, we are left with the meeting not
> > concluding how exactly these calls across the language boundary.
> >
>
> C++ CCed objects don't even all use COM. Maybe the existing
non-nsISupports
> CC infrastructure could be used for Rust objects. If that doesn't work,
> adding explicit support for Rust objects to the CC, like we have for JS,
> doesn't seem like it would be too difficult. The CC used to nominally
> support arbitrary languages.

Non-nsISupports cycle-collected objects were brought up as what we should
do in a way that avoids XPCOM. AFAICT, in this case, the methods need not
be virtual, so if we wanted to use a vtable-based solution for
cross-language calls, we'd need to make stuff virtual because of the
cross-language call mechanism.

In the absence of cross-language inlining (LLVM on both sides), which I'm
told is not about to happen on Windows, the trickery with "this" to make
FFI look C++ish to the C++ caller should inline into a plain function call,
which should be preferable to a virtual call. (If we ever get
cross-language inlining with clang and rustc, the inlining opportunity
would be even better.)

[Ms2ger]

On 22/06/16 19:05, Henri Sivonen wrote:
> We shouldn't expect to be able to use Servo's implementations of DOM
> APIs in a drop-in a manner in Gecko. Because Servo allocates Rust
> objects on the JavaScript heap, but Gecko doesn't allocate C++ objects
> on the JavaScript heap,

For the record, this is not true. The significant difference is that
Servo creates a reflector (JSObject) for any DOM object as soon as it is
created, and this reflector is responsible for the deallocation of the
DOM object. Gecko instead manages the DOM object through reference
counting, and only creates a reflector when JS code needs it.

> the memory management arrangements of the two
> engines are fundamentally different and, therefore, the same WebIDL
> bindings won't work.

... but your conclusion remains correct.

HTH
Ms2ger