Re: [Caml-list] RPC for OCaml?

[rixed]

Note: It has been suggested that I sent this here. This is really addressed

to anyone who've used async_rpc_kernel library.

I'm thinking about designing the ultimate RPC library for OCaml (mostly for

MirageOs) and had a look at this lib and have a few questions.

- The description says that its a "Platform-independant" library. What is

  meant by that?

- There are a lot of dependencies; have someone ever tried to port it to

  MirageOS?

- The handshake (for version negociation) in addition to the TCP handshake

makes short lived connections expensive. Wouldn't merely binding the version

to the port number (in other words, listens to as many ports as supported

versions) be both faster and simpler?

- I've always though streams should be implemented on top of RPC; can I ask

what special purpose you had when you decided to implement streams as a "first

class" protocol?

- You have some custom flow-control on top of TCP flow control. Can I ask why

it was necessary?

- Same question for keep-alive (although I understand that TCP keep alive is

a pain)?

- Regarding the keep-alive, the reception of any message could reset the

timeout (as oposed to only specificaly a keep-alive message). Why this choice?

- You decided to attach a state per connection, that servers can use to

store per client stuff. I am not a big fan of this, as I consider many times

this will be either unused or, in a world where the clients are authentified,

inapropriate (as you want a state per identification in that case not per

socket). Are many of your services actually use this?

- There is a deferred for when a connection stops. This can be used to save

the state I guess. Any other use case?

- I was surprised that a server may have a custom response to unknown incoming

messages. When is this useful for a service to customize this behavior?

- I'm not familiar with sexp_of_t but I guess there is no garantee that an

s-expression that can be unserialized into an ocaml value of a given type was

actually obtained from that type (a char may also be a string of length 1 for

instance). I guess as the received structures get more complex this is less

likelly to happen but still in a networked context the question of "remote"

type checking is interesting. Have you ever though of a way to garantee (or

make more likely) that a received message was indeed serialized by a

compatible version of sexplib from the expected type? In my pet project I'm

adding a hash of the type name + version but that's just meh.

[Yaron Minsky]

On Fri, Jun 17, 2016 at 7:10 AM, rixed <ri...@happyleptic.org> wrote:
> Note: It has been suggested that I sent this here. This is really addressed

By me! I also looped in the async-rpc folk devs explicitly.

> to anyone who've used async_rpc_kernel library.
>
> I'm thinking about designing the ultimate RPC library for OCaml (mostly for
> MirageOs) and had a look at this lib and have a few questions.
>
> - The description says that its a "Platform-independant" library. What is
> meant by that?

Meaning, it doesn't depend on the Unix library. We're actively using
it in Javascript, where we wrote our own little driver for the
scheduler. There's no such for Mirage or Windows as far as I know,
but both should be doable.

> - There are a lot of dependencies; have someone ever tried to port it to
> MirageOS?

Not as far as I know.

> - The handshake (for version negociation) in addition to the TCP handshake
> makes short lived connections expensive. Wouldn't merely binding the version
> to the port number (in other words, listens to as many ports as supported
> versions) be both faster and simpler?

We mostly haven't optimized for the short-lived connection case. The
port thing has its own set of

> - I've always though streams should be implemented on top of RPC;
> can I ask what special purpose you had when you decided to
> implement streams as a "first class" protocol?

Doing it as a first-class "protocol shape" makes sense, since it's a
common use case, and you can't really implement it any other way (RPCs
always have acks, but not so for each message down a pipe). We have
some dreams of using session types to make these shapes less first
class, but that's off in the unknown future.

> - You have some custom flow-control on top of TCP flow control. Can
> I ask why it was necessary?

Not sure.

> - Same question for keep-alive (although I understand that TCP keep alive is
> a pain)?

We've seen lots of issues and bugs with TCP-level keepalive, and so we
built our own.

> - Regarding the keep-alive, the reception of any message could reset
> the timeout (as oposed to only specificaly a keep-alive
> message). Why this choice?

Not sure.

> - You decided to attach a state per connection, that servers can use
> to store per client stuff. I am not a big fan of this, as I
> consider many times this will be either unused or, in a world
> where the clients are authentified, inapropriate (as you want a
> state per identification in that case not per socket). Are many of
> your services actually use this?

Having it chosen per client lets you construct the state any way you
want, so I think you can do it per authenticated user, once you get
your hands on the user.

Anyway, I think we do this a decent amount, but the most common case
is that the state is shared. I see no problem with the API as it
stands, though.

> - There is a deferred for when a connection stops. This can be used
> to save the state I guess. Any other use case?

I'm not sure off-hand, but I imagine it has lots of uses.

> - I was surprised that a server may have a custom response to
> unknown incoming messages. When is this useful for a service to
> customize this behavior?

Absolutely. In some cases you might want to write something to a log
(and there are different forms of logging). Sometimes you might want
to raise an alert to some other system, and in some cases, you might
consider it a serious error that makes you want to close the
connection or even shut down the server.

> - I'm not familiar with sexp_of_t but I guess there is no garantee
> that an s-expression that can be unserialized into an ocaml value
> of a given type was actually obtained from that type (a char may
> also be a string of length 1 for instance). I guess as the
> received structures get more complex this is less likelly to
> happen but still in a networked context the question of "remote"
> type checking is interesting. Have you ever though of a way to
> garantee (or make more likely) that a received message was indeed
> serialized by a compatible version of sexplib from the expected

> type? In my pet project I'm yadding a hash of the type name +

> version but that's just meh.

Async-RPC uses bin-io, not sexplib, but the same question applies. We
have a feature currently under development to add just such a "type
hash" to catch such mistakes. We also have a discipline of what we
call "stable types", where there are explicit versions for different
types, and one generally only uses these explicitly versioned types
when constructing types for use with Async-RPC, so as to avoid
problems when one process upgrades but others remain behind.

y



> --
> You received this message because you are subscribed to the Google Groups
> "ocaml-core" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to ocaml-core+...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Andres Varon]

Hello,

I will add a few things to what Ron has already said:

On Fri, Jun 17, 2016 at 8:44 AM, Yaron Minsky <ymi...@janestreet.com> wrote:

On Fri, Jun 17, 2016 at 7:10 AM, rixed <ri...@happyleptic.org> wrote:


> - The handshake (for version negociation) in addition to the TCP handshake
>   makes short lived connections expensive. Wouldn't merely binding the version
>   to the port number (in other words, listens to as many ports as supported
>   versions) be both faster and simpler?

We mostly haven't optimized for the short-lived connection case.  The
port thing has its own set of

That is a fair solution but in our use cases the operational overhead would not be worth the speedup. One common practice at Jane Street is to use a random port assigned by the operating system as the RPC port of a given running application, and rely on other service discovery mechanism to find it. It would be rather inconvenient if, in addition to whatever naming convention we had to find the service, there was the need to advertise different ports for different rpc versions. It would get even more painful if an application has some fixed ports and we need to roll a new version of the protocol which would require taking more ports.

Ultimately the RPC connection will be used to dispatch "something". I believe that the overhead of the handshake can easily be amortized by packing the relevant information with the first RPC that is dispatched. We have not done any work on that front yet, but the design that we have for the next version of RPC will allow us to do this (this is what Ron referred to below  as "using session types"; the protocol itself will look very different when that lands.)

> - I've always though streams should be implemented on top of RPC;
>   can I ask what special purpose you had when you decided to
>   implement streams as a "first class" protocol?

Doing it as a first-class "protocol shape" makes sense, since it's a
common use case, and you can't really implement it any other way (RPCs
always have acks, but not so for each message down a pipe).  We have
some dreams of using session types to make these shapes less first
class, but that's off in the unknown future.

> - You have some custom flow-control on top of TCP flow control. Can
>   I ask why it was necessary?

Not sure.

What are you referring to here? In general we don't quite have flow control, but perhaps you refer to the specific case of Pipe_rpc's and whether or not a client can push back? I am likely to be missing something obvious here.

If you do refer to the Pipe_rpc case, then the main reason is multiplexing. Most uses of RPC are over TCP connections which guarantee order. There is almost no notion of fairness in Async-RPC, and a fast enough producer can overwhelm the receiving process. On the receiving end of an RPC connection there typically is a buffer from which data is consumed and processed. A producer can deliver sufficiently fast to fill the receiving buffer, effectively stopping any other RPC dispatch over that channel. This bad pattern was common for pipe rpcs and so we introduced flags that could help developers avoid such problem. In particularly bad cases the receiving side may even consider the connection dead as there may be no data flow for long periods of time!

 

> - Regarding the keep-alive, the reception of any message could reset
>   the timeout (as oposed to only specificaly a keep-alive
>   message). Why this choice?

Not sure.

Perhaps the strongest reason is that there is no guarantee that the heartbeats will arrive timely. The library is typically using a single TCP connection to dispatch RPCs and send heartbeats.  As TCP guarantees the order of the messages, then a large number of RPCs can overwhelm the receiving side, causing delays for any heartbeats to arrive.

Ultimately the hearbeat is only treated as evidence that the other side is alive. The reception of any other message serves the same purpose, and so it seems to me like a natural choice. I would have made the same choice had I written that bit of logic back in the day!

 

> - You decided to attach a state per connection, that servers can use
>   to store per client stuff. I am not a big fan of this, as I
>   consider many times this will be either unused or, in a world
>   where the clients are authentified, inapropriate (as you want a
>   state per identification in that case not per socket). Are many of
>   your services actually use this?

Having it chosen per client lets you construct the state any way you
want, so I think you can do it per authenticated user, once you get
your hands on the user.

Anyway, I think we do this a decent amount, but the most common case
is that the state is shared.  I see no problem with the API as it
stands, though.

> - There is a deferred for when a connection stops. This can be used
>   to save the state I guess. Any other use case?

I'm not sure off-hand, but I imagine it has lots of uses.

Even logging! Freeing resources for long lived sessions is another commonly used case.

I hope this helps.

regards,

Andres

[gri...@gmail.com]

> > - The handshake (for version negociation) in addition to the TCP handshake
> > makes short lived connections expensive. Wouldn't merely binding the version
> > to the port number (in other words, listens to as many ports as supported
> > versions) be both faster and simpler?
>
> We mostly haven't optimized for the short-lived connection case. The
> port thing has its own set of
>
> That is a fair solution but in our use cases the operational overhead would not be worth the speedup. One common practice at Jane Street is to use a random port assigned by the operating system as the RPC port of a given running application, and rely on other service discovery mechanism to find it.

I don’t get this. This is precisely because you use a discovery service that having one port per supported version would work.
Of course, the version number has tp be part of the name. So that when a client want to connect to service X, version V, it can be given the IPs and ports of the servers implementing that version of X. Clients have to implement only one version, and you can decommission servers implementing old versions when nobody uses it for a while. This seems so much simpler.

> It would be rather inconvenient if, in addition to whatever naming convention we had to find the service, there was the need to advertise different ports for different rpc versions.

I fail to see what difference it makes. Server X listens on 10.1.2.3:4567 and advertise this socket pair as version V of service X to the discovery service. You probably already do that, just without the version number.

> It would get even more painful if an application has some fixed ports and we need to roll a new version of the protocol which would require taking more ports.

If an application is composed of several (micro) services themselves composed of a few RPC, then indeed starting a new version means starting as many services and new RPC (possibly some RPC will be the same, some may disappear, some new may appear). So? Service inventory is automatic anyway. For the clients, who generally see only a tiny fractions of the RPC, it makes little difference.

Additional benefit: you can monitor/scale independently the various versions.

Of course if you deploy a new version ten times a day, and your application exposes a lot of micro services, then it’s more likely you will have a high number of different versions running simultaneously. If that’s an issue, deprecate them faster? If you use to convert internally from all those versions then you may have trouble to clean up your code from older versions at this rate of change. Running them separately at least solve this issue.

> Doing it as a first-class "protocol shape" makes sense, since it's a
> common use case, and you can't really implement it any other way (RPCs
> always have acks, but not so for each message down a pipe).

You could imagine an RPC returning nothing, and asking for an Ack just from time to time.
(in my pet case, RPC returning unit returns effectively nothing).

> [ flow control ]

>
> What are you referring to here? In general we don't quite have flow control, but perhaps you refer to the specific case of Pipe_rpc's and whether or not a client can push back? I am likely to be missing something obvious here.

client_pushes_back, exactly. It looks like a work around for some unwanted buffering.

> If you do refer to the Pipe_rpc case, then the main reason is multiplexing.
> Most uses of RPC are over TCP connections which guarantee order. There is almost no notion of fairness in Async-RPC, and a fast enough producer can overwhelm the receiving process. On the receiving end of an RPC connection there typically is a buffer from which data is consumed and processed. A producer can deliver sufficiently fast to fill the receiving buffer,

Ah! Similar to the selective receives issue from Erlang.
Using a single socket for unrelated writers or readers is indeed a lot of troubles.
Why not let IP do the multiplexing? Again to save some ports?

> > - Regarding the keep-alive, the reception of any message could reset
> > the timeout (as oposed to only specificaly a keep-alive
> > message). Why this choice?
>
> Not sure.
>
> Perhaps the strongest reason is that there is no guarantee that the heartbeats will arrive timely. The library is typically using a single TCP connection to dispatch RPCs and send heartbeats.
> As TCP guarantees the order of the messages, then a large number of RPCs can overwhelm the receiving side, causing delays for any heartbeats to arrive.

You do not care about the reception of the keep-alive in general. Keep-alives are for the network stack and the network gears in the way, not for the app. And when there are already many messages you should not even send any keep-alive (A normal RPC already does the job of the NOP message which is a keep-alive). That was precisely my question. Health checks (of which heartbeats are the simplest case) are a different story: their purpose is to make sure the other peer is still on, reachable (and, optional, healthy). They are unrelated to the socket you use for the RPC and could use another one.


Thank you for your time and attention!

[David House]

Andres has doubtless ruminated on these issues much more than me, but let me offer a few of my own thoughts:

Keep in mind that one versions individual RPCs, not services. Most of our services implement a handful of RPCs: normally some number for their core purpose, plus a few debugging queries that let one inspect the state of the service.

One could envision a scheme where a server listens on a separate port for each (rpc*version) pair that it supports. This would indeed probably work pretty well. I can imagine the following issues with that scheme:

- Ordering guarantees. Using a single socket means that you can send RPC A, then send RPC B, and you have the guarantee that A will arrive before B. This is probably the most important reason. Mostly different RPCs are semantically independent but not always.

- Async RPC works over any reader/writer interface, not just socket. Another use case is for one process to fork/exec another binary, and then use async-rpc to communicate with it over its stdin and stdout. The everything-through-one-channel communication method generalises more easily to other transports.

- Lack of ports. 65k might sound like a lot, and I doubt we would hit the limit often, but I would be worried that some edge cases where one was running many many processes on a single box might bump up into this restriction.

- Client RPCs. Async-rpc is actually symmetric once the connection is established. Either end may call RPCs on the other. It would be necessary for the server to reach back and open a new connection on the client machine in order to do this, which is more complicated, and interacts less well with firewalls which often only allow TCP connections to be established in one direction.

Also, note that the versioning scheme you describe is only one of three that async rpc supports -- we call it "callee converts"; "caller converts" and "both convert" are also possible. "Caller converts" is useful when you have, for instance, many instances of a given application running across your estate, and you want to have a single "commander" binary that can query those servers for some state. It's natural to make sure the commander can speak all of the versions of the deployed servers, and only worry about implementing a single version in the server itself. You can look at the mli for versioned_rpc.mli for more details. This is doable in your scheme as well but you need a more complex service discovery (you'd need to be able to ask which version is associated with a given port).

More generally: there are advantages to your scheme, for sure. Not having to do things like flow control, multiplexing and heartbeating ourselves would be neat. But I would also worry about the loss of flexibility by handing off these parts to the networking layer. TCP/IP is pretty well designed but it can be hard to customise its behaviour on an application-by-application basis, and there are some rough edges in there that would probably make life difficult sometimes (eg limited number of ports). Doing these things at a higher level means you get a lot more power.