Unix domain socket I/O in chrome JS

[Philipp von Weitershausen]

The need for talking to Unix domain sockets from Gecko has come up
several times now:

* Radio: for talking to rild we invented
https://mxr.mozilla.org/mozilla-central/source/ipc/ril/Ril.cpp and
https://mxr.mozilla.org/mozilla-central/source/dom/system/gonk/SystemWorkerManager.cpp
* USB volumes: for talking to vold we created
https://mxr.mozilla.org/mozilla-central/source/dom/system/gonk/VolumeManager.cpp
* USB/Wifi tethering will require talking to a daemon over Unix domain socket
* AFAIK bluetooth may have a need for it, too (I'll let qDot chime in himself)
* can people think of other uses?

We're thinking of exposing this functionality directly to JS, thus
eliminating the need for lots of C++ boilerplate code. Since this
stuff currently happens on IO / IPC threads, I propose we only expose
the JS API to ChromeWorkers. The ril_worker would then directly talk
to rild/rilproxy. The VolumeManager could be reimplemented as a
ChromeWorker. The tethering stuff could be added to the net_worker.
You get the picture.

Here's an API proposal based on an earlier one by cjones:

interface ChromeWorker {
DOMRequest openSocket(String addr);
};

interface Socket {
void send(byte[] data); // byte[] can be regular Array or Uint8Array
attribute Function onrecv; // gets Uint8Array
attribute Function onclose;
attribute Function onerror;
};

Example:

let req = openSocket("unix:/dev/socket/netd");
req.onerror = ...;
req.onsuccess = function onsuccess() {
let sock = req.result;
sock.onrecv = ...;
sock.onclose = ...;
sock.onerror = ...;
sock.send("start\n"); };
};

What do folks think?

[Dave Hylands]

Just thinking out loud about your proposed API.

The synchronous send is probably ok for command/response type protocols (provided the buffers are larger than your typical command).

For more unidirectional type streams (data >> buffer-size), you probably want an onempty or onsendcomplete or onspaceavail type callback so that you have the ability to flow-control in the sending direction without having to block the sender. You may also want a async send that returns the number of bytes actually written.

You'll also want to have interfaces which allow sending/receiving arbitrary binary data. In particular, vold uses a binary zero character to delimit commands/responses.

Dave Hylands

> _______________________________________________
> dev-b2g mailing list
> dev...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-b2g
>

[Philipp von Weitershausen]

Thanks for the feedback!

On Thu, May 31, 2012 at 6:12 PM, Dave Hylands <dhyl...@mozilla.com> wrote:
> The synchronous send is probably ok for command/response type protocols (provided the buffers are larger than your typical command).

(or we handle the fact that we have to write the data in chunks under
the covers.)

> For more unidirectional type streams (data >> buffer-size), you probably want an onempty or onsendcomplete or onspaceavail type callback so that you have the ability to flow-control in the sending direction without having to block the sender. You may also want a async send that returns the number of bytes actually written.

All this makes sense if we need this kind of control-flow in JS. I'm
happy to add it if that's the case; I was just aiming for the most
minimal thing we need atm.

> You'll also want to have interfaces which allow sending/receiving arbitrary binary data.

That's what this is.

[Fabrice Desre]

On 05/31/2012 06:00 PM, Philipp von Weitershausen wrote:
> The need for talking to Unix domain sockets from Gecko has come up
> several times now:

> ...
>
> What do folks think?

Do we need something different from the TCPSocket API (except for the
creation part) ?

Fabrice
--
Fabrice Desré
b2g team
Mozilla Corporation

[Philipp von Weitershausen]

On Thu, May 31, 2012 at 11:30 PM, Fabrice Desre <fab...@mozilla.com> wrote:
> On 05/31/2012 06:00 PM, Philipp von Weitershausen wrote:
>>
>> The need for talking to Unix domain sockets from Gecko has come up
>> several times now:
>> ...
>>
>> What do folks think?
>
>
> Do we need something different from the TCPSocket API (except for the
> creation part) ?

Great suggestion! We can definitely model the API more closely after
TCPSocket. This would essentially mean s/onrecv/ondata/. We could even
borrow the ondrain and bufferedAmount bits to give us what dhylands
was suggesting.

(On a general note: I agree that similar/same APIs are nice, but keep
in mind that this is (a) an internal API (as far as my proposal goes,
it will *only* be available in ChromeWorkers) and (b) TCPSocket can
change during standardization, which means the two APIs might get out
of sync again. So I don't thik we need to try *too* hard.)

[Donovan Preston]

On Thu May 31 23:57:01 2012, Philipp von Weitershausen wrote:
> On Thu, May 31, 2012 at 11:30 PM, Fabrice Desre <fab...@mozilla.com> wrote:
> Great suggestion! We can definitely model the API more closely after
> TCPSocket. This would essentially mean s/onrecv/ondata/. We could even
> borrow the ondrain and bufferedAmount bits to give us what dhylands
> was suggesting.

This is a nice idea. Having a compatible interface for both tcp sockets
and unix domain sockets would be nice so that code written for one
could be used on the other type of transport.

However in b2g's case, the unix domain sockets are probably only going
to be for internal platform use, so it's not really that important.

(Plus, does anyone really write code which would make sense to run over
both types of transport? For unix daemons running on servers lots of
people log in to it makes sense, but it's less obvious how it would be
useful for b2g)

[Philipp von Weitershausen]

On Jun 1, 2012 10:37 AM, "Donovan Preston" <dpre...@mozilla.com> wrote:
>
> On Thu May 31 23:57:01 2012, Philipp von Weitershausen wrote:
>>
>> On Thu, May 31, 2012 at 11:30 PM, Fabrice Desre <fab...@mozilla.com>
wrote:
>> Great suggestion! We can definitely model the API more closely after
>> TCPSocket. This would essentially mean s/onrecv/ondata/. We could even
>> borrow the ondrain and bufferedAmount bits to give us what dhylands
>> was suggesting.
>
>
> This is a nice idea. Having a compatible interface for both tcp sockets
and unix domain sockets would be nice so that code written for one could be
used on the other type of transport.

Except that the use cases are very different. One is for off-thread socket
I/O in chrome. The other is building webapps that need TCP sockets. I can't
really see an intersection.

> However in b2g's case, the unix domain sockets are probably only going to
be for internal platform use, so it's not really that important.
>
> (Plus, does anyone really write code which would make sense to run over
both types of transport? For unix daemons running on servers lots of people
log in to it makes sense, but it's less obvious how it would be useful for
b2g)

Correct (on both points)

[Philipp von Weitershausen]

Filed https://bugzilla.mozilla.org/show_bug.cgi?id=760649, going to
work on this.

On Thu, May 31, 2012 at 6:00 PM, Philipp von Weitershausen
<phil...@googlemail.com> wrote:
> The need for talking to Unix domain sockets from Gecko has come up
> several times now:
>

[Robert Kaiser]

Philipp von Weitershausen schrieb:

> We're thinking of exposing this functionality directly to JS

The significant problem I'm seeing is that anything exposed to web app
JS (chrome JS is a different beast there) is supposed to have a goal of
being standardized and implemented as a part of the generic web
platform. It might be a problem to implement access to unix domain
sockets cross-platform on different web browsers.
Also, I fear that this ends up in a lot of non-portable and therefore
not-of-the-web code.

Robert Kaiser

[Philipp von Weitershausen]

The goal is not to make this feature accessible to web JS. In fact,
that's an explicit non-goal. My plan, as I stated in my original
email, was to only expose it to *ChromeWorkers*, which means (a) you
can only do from Chrome code and (b) you can't do it on the main
thread.

[Robert Kaiser]

Philipp von Weitershausen schrieb:

> The goal is not to make this feature accessible to web JS. In fact,
> that's an explicit non-goal. My plan, as I stated in my original
> email, was to only expose it to *ChromeWorkers*, which means (a) you
> can only do from Chrome code and (b) you can't do it on the main
> thread.

OK, that sounds good. I was confused there because almost every use of
"implement in JS" in the B2G space is about the web apps running on the
B2G system. Implementation of B2G interfaces to the web apps is a
different topic of course and I don't see a problem there.

Robert Kaiser