JS-ctypes Support and WebExtensions

[ganjo...@gmail.com]

Hi,

As binary XPCOM is no longer being supported in Firefox 41 and any releases after that, one of the suggested alternatives was to use JS-ctypes. I am currently in the process of updating my extension to use JS-ctypes instead.

I would like to know how much longer JS-ctypes will be supported. Can we expect the support to continue for the next few releases as well as the next ESR release (FF 52)? I would also like to know if it will be required to be updated for each new Firefox release and also regarding backward compatibility with older versions if an update is required.

I am aware that Firefox will be moving on to WebExtensions in the future. Will there be a method of running and invoking native C/C++ codes in WebExtensions?

Thanks!

[Benjamin Smedberg]

JS-ctypes is available, but it is *not* the recommended solution for
interfacing with binary code. It is prone to memory/GC hazards which are
not obvious on code inspection.

The recommended solution right now is to communicate with an external
binary over a pipe using system/child_process from the addon SDK.
https://developer.mozilla.org/en-US/Add-ons/SDK/Low-Level_APIs/system_child_process

In the near future we will be adding support for native messaging to
WebExtensions and that will be the long-term solution.

--BDS

> _______________________________________________
> dev-extensions mailing list
> dev-ext...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-extensions
>

[Andrew McKay]

The tacking bug for chrome.runtime.connectNative is here:
https://bugzilla.mozilla.org/show_bug.cgi?id=1190682

On Wed, Apr 20, 2016 at 7:00 AM, Benjamin Smedberg
<benj...@smedbergs.us> wrote:
> JS-ctypes is available, but it is *not* the recommended solution for
> interfacing with binary code. It is prone to memory/GC hazards which are
> not obvious on code inspection.
>
> The recommended solution right now is to communicate with an external
> binary over a pipe using system/child_process from the addon SDK.
> https://developer.mozilla.org/en-US/Add-ons/SDK/Low-Level_APIs/system_child_process
>
> In the near future we will be adding support for native messaging to
> WebExtensions and that will be the long-term solution.
>
> --BDS
>
> On Wed, Apr 20, 2016 at 2:15 AM, <ganjo...@gmail.com> wrote:
>

[ganjo...@gmail.com]

On Wednesday, April 20, 2016 at 10:00:47 PM UTC+8, Benjamin Smedberg wrote:
> JS-ctypes is available, but it is *not* the recommended solution for
> interfacing with binary code. It is prone to memory/GC hazards which are
> not obvious on code inspection.
>
> The recommended solution right now is to communicate with an external
> binary over a pipe using system/child_process from the addon SDK.
> https://developer.mozilla.org/en-US/Add-ons/SDK/Low-Level_APIs/system_child_process
>
> In the near future we will be adding support for native messaging to
> WebExtensions and that will be the long-term solution.
>
> --BDS
>

> On Wed, Apr 20, 2016 at 2:15 AM, Joshua Gan wrote:
>
> > Hi,
> >
> > As binary XPCOM is no longer being supported in Firefox 41 and any
> > releases after that, one of the suggested alternatives was to use
> > JS-ctypes. I am currently in the process of updating my extension to use
> > JS-ctypes instead.
> >
> > I would like to know how much longer JS-ctypes will be supported. Can we
> > expect the support to continue for the next few releases as well as the
> > next ESR release (FF 52)? I would also like to know if it will be required
> > to be updated for each new Firefox release and also regarding backward
> > compatibility with older versions if an update is required.
> >
> > I am aware that Firefox will be moving on to WebExtensions in the future.
> > Will there be a method of running and invoking native C/C++ codes in
> > WebExtensions?
> >
> > Thanks!
> > _______________________________________________
> > dev-extensions mailing list
> > dev-ext...@lists.mozilla.org
> > https://lists.mozilla.org/listinfo/dev-extensions
> >

Hi Benjamin,

Thank you for the reply. Is there a timeline for how long js-ctypes will still be supported? Regarding the memory/GC issues, can you elaborate on this please? Where do these hazards usually occur? Thanks!

Joshua

[Benjamin Smedberg]

JS ctypes will not be available to webextensions. We are on a path so that
all Firefox extensions will be webextensions, and so at some point ctypes
will not be available. We don't have a firm timeline for that process.

The GC hazard is typically something like this:

function outptr() {
let intvalue = ctypes.int64_t(0);
let intptr = intvalue.address();
return intptr;
}
// call a function of the form bool string_to_int64(char* s, int64_t*
result)
let result = outptr();
let ok = string_to_int64("123467789", result);
let value = result.contents();

This is not safe code because the pointer can live longer than the thing
it's pointing to. This is especially true if there is anything asynchronous
about the code: if you're using callback functions you have to make sure
that *everything* is rooted across the call.

Also, you're marshalling data types mostly by hand. It's just not a
reliable long-term engineering solution for most things.

--BDS

[Noit]

Aw darn. Ctypes was pretty awesome. Off mainthread. Much easier then making binaries for the different platforms (especially when needing just the platform APIs). It's easy to VM the other platforms, but not easy to get all the tools (and knowledge of the tools) for each platform to make the binaries.

Those invisible mem hazards though, yeah tricky stuff.

I'll be hoping though, even if slim chances, mayyyybe it will be allowed somewhere in the future, haha. I know some pretty popular native apps via js rely on FFI's like of js-ctypes - ie: https://www.trueinteractions.com/ . Also I know some platforms APIs require to be on the mainthread of the target app (Firefox), I'm not sure if that can be accomplished with the chrome.runtime.connectNative or child_process methods.


Anyways - to the OP, if you do want to go ctypes for the time being there is a lot of the stuff already typed up for Mac, Windows, and Unix/Linux here: https://github.com/Noitidart/ostypes

Demos here: https://github.com/Noitidart/ostypes_playground/