request for comment: cross-boundary messaging API

[Istvan Soos]

Hi Dartisans,

I've drafted a high-level messaging API that could work in various
environments that need some kind of boundary cross (wire format and
object serialization, e.g. between web- and service workers, web
sockets...). My desire is that instead of sending a blob through
postMessage() and rolling your own parsing on the other side, most
applications could define their own data types and communication model
in a more apparent way.

The main idea is that each side of the pipe provides a 'hub':
https://github.com/isoos/message_hub/blob/master/lib/message_hub.dart#L5

You register adapters that handle the serialization and deserialization:
https://github.com/isoos/message_hub/blob/master/lib/message_hub.dart#L7

You register message handlers (a service that responds to requests):
https://github.com/isoos/message_hub/blob/master/lib/message_hub.dart#L16

And the rest of the API is either generic stream (sendMessage,
onMessage) or request-reply model (call, invoke).

I've also drafted three "transport handler" for web_worker, web_socket
and service_worker, which gives you an idea how this may work on
different transports:
https://github.com/isoos/message_hub/tree/master/lib

Please, please, please, let me know what you think about it! I'd like
to have a polished API that may be useful for different projects. It
is not used anywhere yet, and it is possible to change it in any way
that provides an improvement.

Thanks,
Istvan

[jan.m...@gmail.com]

For the cross-wire format, wouldn't gRPC be a good option?

Please, please, please, let me know what you think about it! I'd like
to have a polished API that may be useful for different projects. It
is not used anywhere yet, and it is possible to change it in any way
that provides an improvement.

Thanks,
   Istvan

-- 
For other discussions, see https://groups.google.com/a/dartlang.org/

For HOWTO questions, visit http://stackoverflow.com/tags/dart

To file a bug report or feature request, go to http://www.dartbug.com/new
--- 
You received this message because you are subscribed to the Google Groups "Dart Misc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to misc+uns...@dartlang.org.

[Istvan Soos]

On Mon, Mar 20, 2017 at 4:47 PM, <jan.m...@gmail.com> wrote:
> For the cross-wire format, wouldn't gRPC be a good option?

Interesting suggestion, but I'm not sure the scope of it. What part of
the gRPC wire format do you refer to as adopting? AFAICT it is mostly
HTTP2 headers + protobuf binary (or JSON proxy for REST-like
services).

Nevertheless, it would be interesting to see if we could use the `rpc`
package (or any of the similar rest-like codegen packages) on pub to
define the API and generate code for various environments...

Thanks,
Istvan

[Florian Loitsch]

I didn't have the time to look in detail, but this looks similar to what I tried to do with Mojo a year ago. [0]

Mojo has since been deprecated and been replaced with fidl [0], and someone (I?) would need some time and effort to update my work from then. From what I have heard the API hasn't changed that much, so I guess it wouldn't be too bad.

Contrary to your (Istvan) approach, fidl requires a separate interface language and a compiler. In return you get the advantage of fitting into Fuchsia and lots of tools, like IDE support.

[0] https://docs.google.com/document/d/1IVr0X8UkDE1tOyiDMJsBKx_wTspO_IlBLHqdgGJdHro

[1] https://fuchsia.googlesource.com/fidl/

[Istvan Soos]

Thanks Florian,

I don't mind using a separate interface language and compiler, so fidl
is a great candidate for using it here.

I've been digging into what others are using, I've encountered WAMP.
It is a WebSocket sup-protocol, with all of the features I've been
looking for, and it has support on many platforms:
http://wamp-proto.org/

Does anybody have experience with it?

Thanks,
Istvan

[Istvan Soos]

After a few more iterations, I've came up with a version that can act
as a low-level transport layer for a higher-level API like fidl or
other RPC that gets compiled from an IDL.

This is my cross-isolate example with complex objects that need some
serialization:
https://github.com/isoos/message_hub/blob/master/example/isolate_factorization.dart

There is a simpler echo service example alongside. If you have spare
time during IO, please take a look and let me know what you think
about it (and the direction I was taking).

Thanks,
Istvan

[Florian Loitsch]

Looks good.

You could consider letting the caller (the one that initiates everything) to be able to set up the callee. Something like:

mainHub.bindService(

  "factorization",

  const FacturizationService());

class FacturizationService implements HubService {

  const FacturizationService();

  void run() { ... }

  Map encodeRequest(Request x) { ... }

  Request decodeRequest(...) { ... }

  Map encodeResponse(Response x) { ... }

  Response decodeResponse(...) { ... }

}

This way, you could dispatch to an isolate without needing lots of work on each side. Basically, you could spawn a generic isolate, and then register services from your side. This would keep the code a bit more grouped.

[Natalie Weizenbaum]

This seems pretty similar in concept to the stream channel API—have you looked at that at all?

[Istvan Soos]

Yes I did, and I've added a thin wrapper for using it:
https://github.com/isoos/message_hub/blob/master/lib/stream_channel/stream_channel.dart

The short reason I hadn't built everything on top of stream_channel: I
found it hard to apply it on top of traditional message queues, where
the distributed nature of the queue prevents direct two-way channel,
and may require to rely on the queue's filter mechanism for e.g.
in-reply-to ids. Hence the different kind of API.

Istvan

[Istvan Soos]

Hm, interesting idea, although the caller-initiated setup is feasible
only for VM isolates, and web-workers or other protocols wouldn't be
able to use it. On the other hand, bundling the caller and the handler
has code organization advantages, and which may be preferable in other
scenarios too. I'll play with it.

Thanks,
Istvan


On Wed, May 17, 2017 at 5:01 PM, 'Florian Loitsch' via Dart Misc

[Florian Loitsch]

On Mon, May 22, 2017 at 11:48 AM, Istvan Soos <istva...@gmail.com> wrote:

Hm, interesting idea, although the caller-initiated setup is feasible
only for VM isolates, and web-workers or other protocols wouldn't be
able to use it. On the other hand, bundling the caller and the handler
has code organization advantages, and which may be preferable in other
scenarios too. I'll play with it.

I think webworkers should work as well. I spend some time a few years back, to be able to transfer normal objects in dart2js as well.

Both sides have the limitation that you can't share closures, though.

Dart2js furthermore has the bug that it doesn't canonicalize transferred const objects. But it transfers them and allocates a new (similar) object on the other side.

--

Give a man a fire and he's warm for the whole day,
but set fire to him and he's warm for the rest of his life. - Terry Pratchett