Request: RPC over ports

[Brian Hicks]

General Problem: It's really easy to send or receive messages to JavaScript via ports. But if you need to make a roundtrip to JavaScript, your life suddenly gets much harder since there is no request/response mechanism.

In other words, you can send to or receive from JavaScript, but you fall into a hole if you need to do both.

                Receiving     No Receiving

            ┌───────────────┬──────────────┐

    Sending │      ???      |   Cmd msg    │

            ├───────────────┼──────────────┘

 No Sending │   Sub msg     │

            └───────────────┘

Specific Use-Cases: My team needs a read-through cache on the client side, in which we check the localStorage cache through ports and make an HTTP request if it's empty. Right now we have two options: 

1. Make our own request/response mechanism on top of the Elm Architecture. We don't want to do this because it would create complexities on top of TEA that we don't want to maintain long-term.
2. Move the HTTP read into JavaScript. We don't want to do this because we would rather write in Elm, where we can trust the compiler to have our back and where we can represent these effects as data.

I think this generalizes onto any situation where you need information from the browser after program initialization (excluding packages that already exist like elm-lang/navigation.) That said, I've run into this problem several times but this is the first time where not having something like this has meant we decided not to implement a feature because the solutions were too difficult in Elm.

Background Materials: If you kinda squint at what I'm asking for, it's basically RPC. So there's a bunch of previous material other systems. Things I think are particularly relevant:

• ØMQ did a lot of interesting things with combinations of sockets. Right now we have the equivalent of their PUB and SUB. I'm asking for the equivalent of REQ on the Elm side and REP on the JavaScript side.
• grpc has two modifiers on their sockets: streaming, and directionality. To get what we have, you would specify a unidirectional stream.
• Erlang's gen_server behavior does nice things with regards to setting up explicit messages to be passed back and forth between server and client.

There are also XML-RPC, JSON-RPC, and SOAP. They're all good and bad in their own ways, but share a theme of defining data types that the client and server can agree on. I think changing the data types allowable in ports is out of scope for this request, so they're not as relevant.

There's something in common with all these: they are two parties talking over an unreliable network. I think we need to treat JavaScript the same way. Set timeouts and offer clear visibility into what errors look like.

Suggested Approach:

rpc port name : out1 -> out2 -> Task RPCError res

Where out1 through outN follow the same semantics as outgoing ports, and res follows the same semantics as incoming ports.

RPCError takes care of various failure cases on the JS side: Timeout | Exception String | BadReturnValue String.

On the JavaScript side, an RPC port would be registered on an Elm app on the ports object, with a single method. (Let's call it respondWith to avoid bike shedding here, but the name could certainly be better.)

ports.{name}.respondWith behaves like ports.{name}.subscribe when provided a callback. The difference here: we handle exceptional behavior, set a timeout, and only use the last provided callback. The returned value is treated the same way as a call to ports.{name}.send.

Alternatives: I'm actually having trouble coming up with conceptual alternatives to this proposal. I can think of plenty of ways to implement this, but the diagram above pretty clearly shows where the hole is and I'm not sure of another way to view the problem such that alternate solutions become clear.

That said, here are some alternate implementations I considered:

• use an effect manager to manage the state transitions with multiple ports such that myRpc : reqPort -> respPort -> args -> Task Error resp. Basically gen_server for Elm. The problem here: JavaScript is required to keep track of a request ID in the same way gen_server does under the covers. The types could also turn out pretty funky and not user-friendly. Aside from these things, this could be implemented with Elm 0.18.
• adding responses to outgoing ports. This ends up not being a good idea, since it's quite valid to just send a thing to JavaScript and expect to never get a response. This would make that pattern harder.
• adding requests to incoming ports. This has the same problems, and would preclude receiving a stream of values from an incoming port.
• compose my effects in JavaScript instead of Elm, as outlined above.

Known Issues: 

• The RPC pattern makes it easier to wrap JavaScript libraries instead of writing new, more sensible Elm packages.
• It adds a third option for ports, which could feel like a duplication of functionality. It could make it harder for a beginner to learn Elm, and may be harder for experts to optimize.
• This is not actual network RPC, so some of the strategies in published literature do not apply.

Justification:

• Less painful "glue" code for external runtime APIs. Instead of needing multiple ports and special state handling in the model, we compose tasks. We could write less JavaScript because composing external effects in Elm is now possible.
• Port failure is no longer surprising or hard to debug. We now must be explicit about all kinds of failure that can happen in JavaScript, including missing expected messages.
• Better integration with JavaScript without introducing FFI. If/when Elm ports to a different runtime, the ports will all follow the same mechanic.

Meta Note: I had a conversation about roughly this proposal with someone at elm-conf. But it was really hectic and I don't remember who! I just remember being really really excited about this idea! Sorry if it's a duplicate of something somewhere else; I can't find any prior work in the various Elm fora.

[Evan Czaplicki]

I didn't read everything here yet. I have discussed "task ports" with folks on a couple occasions and want to share some notes first:

A "task port" is a traditional FFI.

Say you want to guarantee that all Haskell FFI bindings are valid types. You could require that they all return (IO a) to capture that effects may occur. That is what this proposal seems to be. When you have a traditional FFI, you push people towards copying JS APIs directly because "those are the functions they gave us". Now we have someone who does 100% copy bindings to some JS library. "It works! It is ready to share!" But you cannot publish code with ports. "Man, we should really allow that." And now we are back to the exact same kernel code discussion.

Given this analysis, should I read things in more detail?

--
You received this message because you are subscribed to the Google Groups "elm-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elm-dev+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elm-dev/0B336033-C511-49CA-8403-E00975CB9C30%40brianthicks.com.
For more options, visit https://groups.google.com/d/optout.

[Brian Hicks]

The difference here: I want to treat JS code as remote. Set timeouts, etc. Treat it as if it's a long way away.

I share your concern about making the interop easy enough that it causes other issues. If approaching it as network communication is insufficient, probably don't waste your time digesting the rest.

To unsubscribe from this group and stop receiving emails from it, send an email to elm-dev+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elm-dev/CAF7GuPEHaR%3DvbrbS8tknev_WvGi3mh5oCCF5rHb43pyLqHi39g%40mail.gmail.com.