Novice OCaml RPC system

[Kenneth Miller]

Is there any interest in anybody either critiquing or adding to my existing library that does a sort of RPC for ocaml using a combination of piqi and ZMQ?

I'd like what you all like: high modularity of each of the components. The RPC system should be separate from the transport mechanism as should the mechanism to generate mappings between the target ocaml functions for external languages to call. But I don't think that I'm at this point yet. Is anybody interested in this?

[Anton Lavrik]

There is a plan to do Piqi-RPC for OCaml, somewhat similar to Piqi-RPC for Erlang [1] but more generalized and transport-independent version of it.

Moreover, the idea is to make this functionality available as a part of piqi-ocaml [2]. ZMQ would be a natural transport choice for practical OCaml applications, but the way I see it, such transport support would need to be maintained as a separate package, e.g. piqi-rpc-ocaml-zmq.  However, if all basic pieces are in place, adding and maintaining transport support should be really easy.

Several things need to happen to make it all possible:

1. Support for a new piqi-rpc binary framing format. Just pushed the first draft for it here: https://github.com/alavrik/piqi/blob/rpc/src/piqi_rpc_v1.piqi

Note that the format is designed in a special way so that there is a direct mapping between HTTP headers and piqi-rpc packet header. It is not relevant for other transports but allows seamless reverse-proxying between HTTP and pipe or ZQM.

2. Stub generator as a part of piqic-ocaml [2]. I am thinking of "piqic-ocaml --rpc" generating <mod>_piqi_rpc.ml in addition to other artifacts. This is similar to what "piqic-erlang-rpc" does. Check Piqi-RPC manual [1] and "piqic-erlang-rpc" source code here: https://github.com/alavrik/piqi-rpc/blob/master/src/piqic_erlang_rpc.erl

3. Transport-independent runtime for generated stubs. There is no good example right now, because piqi-rpc for Erlang presently supports only HTTP and there is no clear separation. But this should give you a basic idea: https://github.com/alavrik/piqi-rpc/blob/master/src/piqi_rpc_runtime.erl

4. Transport support. The idea is to support pipe by default and maintain other transports separately. Here's an example of how transport-level support functions are implemented for the legacy piqi-rpc framing format: https://github.com/alavrik/piqi/blob/rpc/src/piqi_rpc.ml

Note that, in this example, they work backwards in a sense that server code is calling these functions. If we want to do it right, we need transport layer to call into transport-independent runtime and generated stubs instead.

Anton

[1] https://github.com/alavrik/piqi-rpc

[2] https://github.com/alavrik/piqi-ocaml

P.S. Erlang users: once we take care of piqi-rpc for OCaml, we'll go back and revise piqi-rpc for Erlang to follow this new setup.

On Tue, Nov 25, 2014 at 10:06 PM, Kenneth Miller <kennetha...@gmail.com> wrote:

Is there any interest in anybody either critiquing or adding to my existing library that does a sort of RPC for ocaml using a combination of piqi and ZMQ?

I'd like what you all like: high modularity of each of the components. The RPC system should be separate from the transport mechanism as should the mechanism to generate mappings between the target ocaml functions for external languages to call. But I don't think that I'm at this point yet. Is anybody interested in this?

--
You received this message because you are subscribed to the Google Groups "piqi" group.
To unsubscribe from this group and stop receiving emails from it, send an email to piqi+uns...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Motiejus Jakštys]

On Mon, Dec 1, 2014 at 1:50 PM, Anton Lavrik <ala...@piqi.org> wrote:
> There is a plan to do Piqi-RPC for OCaml, somewhat similar to Piqi-RPC for
> Erlang [1] but more generalized and transport-independent version of it.
>
> Moreover, the idea is to make this functionality available as a part of
> piqi-ocaml [2]. ZMQ would be a natural transport choice for practical OCaml
> applications, but the way I see it, such transport support would need to be
> maintained as a separate package, e.g. piqi-rpc-ocaml-zmq. However, if all
> basic pieces are in place, adding and maintaining transport support should
> be really easy.
>
> Several things need to happen to make it all possible:
>
> 1. Support for a new piqi-rpc binary framing format. Just pushed the first
> draft for it here:
> https://github.com/alavrik/piqi/blob/rpc/src/piqi_rpc_v1.piqi

Hi,

thanks for a formal RPC schema! I've been longing for this for a while now.

If I intend to use piqi-rpc over plain TCP socket, I would use
piqi-rpc. Because it defines request, response, errors and payload
type. Good, I don't need to invent these things for every transport
protocol.

For example, take an obscure use case: you would want to use the same
logical interface for external (JSON-HTTP) and internal
(PROTOBUF-SCTP) cases. "new" piqi-rpc would be a first-class support
only for the former interface, because RPC draft is very HTTP
oriented.

How about abstracting the RPC semantics from the transport? I.e. have like:

* piqi-rpc <-- header, request, response, errors, payload type.
* piqi-rpc-http <-- piqi-rpc + error-code + http-header.
* piqi-rpc-tcp <-- piqi-rpc + some custom schema to put error codes in.

piqi-rpc would only be part of piqi without the notion of HTTP.
piqi-rpc-http should extend or re-use piqi-rpc schema, but live in an
independent repository.

Of course it raises all sorts of design questions like "where is the
boundary" and "error reporting really depends on protocol". These are
hard to answer, but maybe with some discussions the boundary can be
found?

I hope it all makes sense; I typed this whole email in a hurry before
rushing out for lunch.

Motiejus

[Anton Lavrik]

Not at all. Why do you think it is?

Framing format as it is defined in the draft can be used over TCP or pipe right away. No HTTP needed.

I did reuse some of HTTP syntax and semantics in the spec. This is convenient and allows us to achieve easy 1-to-1 mapping between piqi-rpc-http header (as in the current version) and piqi-rpc abstract header. The only non-straightforward part is understanding that HTTP status is a combination of piqi-rpc status and piqi-rpc error-code, but there is still 1-to-1 correspondence between them.

Note that, by definition, there is a 1-to-1 mapping between piqi-rpc-binary header and piqi-rpc abstract header, because binary header is just a protobuf-serialized abstract header.

Finally, body format is exactly the same across piqi-rpc-binary and piqi-rpc-http.

Of course it raises all sorts of design questions like "where is the
boundary" and "error reporting really depends on protocol". These are
hard to answer, but maybe with some discussions the boundary can be
found?

Error reporting may depend on protocol, like in case of piqi-rpc-http, because we decided to reuse some of HTTP level 7 functions instead of just using as a l3 transport. However, HTTP is fairly unique in this respect. Using RPC over HTTP is a very practical and natural choice. I can't think of any other l7 protocol for which we may want to provide direct mapping.

On the other hand, l3 error reporting is generally pretty useless for application purposes. Errors typically boil down to "can't connect", "connection lost" or something along these lines. Clients may know better how to handle these and stateless RPC servers generally don't care.

Or am I missing something?

Anton

[Motiejus Jakštys]

Hi, Anton,

reviving an old thread; replies inline.

On Mon, Dec 1, 2014 at 3:19 PM, Anton Lavrik <ala...@piqi.org> wrote:
> On Mon, Dec 1, 2014 at 5:15 AM, Motiejus Jakštys <desir...@gmail.com>
> wrote:
>>
>> For example, take an obscure use case: you would want to use the same
>> logical interface for external (JSON-HTTP) and internal
>> (PROTOBUF-SCTP) cases. "new" piqi-rpc would be a first-class support
>> only for the former interface, because RPC draft is very HTTP
>> oriented.
>
> Not at all. Why do you think it is?

Because of http-header field in the packet-header record[1]. But I
guess I`m just misunderstanding something.

> I did reuse some of HTTP syntax and semantics in the spec. This is
> convenient and allows us to achieve easy 1-to-1 mapping between
> piqi-rpc-http header (as in the current version) and piqi-rpc abstract
> header. The only non-straightforward part is understanding that HTTP status
> is a combination of piqi-rpc status and piqi-rpc error-code, but there is
> still 1-to-1 correspondence between them.

Is piqi-rpc abstract header you are referring to is this whole record[2]?

> Note that, by definition, there is a 1-to-1 mapping between piqi-rpc-binary
> header and piqi-rpc abstract header, because binary header is just a
> protobuf-serialized abstract header.

If the answer to my question above is yes, then why is
'http-header'[1] in the abstract piqi-rpc packet header?

> Finally, body format is exactly the same across piqi-rpc-binary and
> piqi-rpc-http.

OK.

> Error reporting may depend on protocol, like in case of piqi-rpc-http,
> because we decided to reuse some of HTTP level 7 functions instead of just
> using as a l3 transport. However, HTTP is fairly unique in this respect.
> Using RPC over HTTP is a very practical and natural choice. I can't think of
> any other l7 protocol for which we may want to provide direct mapping.
>
> On the other hand, l3 error reporting is generally pretty useless for
> application purposes. Errors typically boil down to "can't connect",
> "connection lost" or something along these lines. Clients may know better
> how to handle these and stateless RPC servers generally don't care.

I have in mind something I used to work with in the past: each service
exposed two kinds of endpoints:
1. piqi-rpc over HTTP.
2. distributed erlang: {ok | error, Payload :: binary()} for a response.

Distributed Erlang has tuples, lists and binaries, which can have
extra information about the errors, very much like in HTTP (e.g. by
adding another element to the return tuple). To be honest, I don't
think it's a good idea to use that protocol, but it gives a nice L7
example transport to think about.

> Or am I missing something?

I feel like I`m missing the point of [1] in that record. To
reiterate, does that field make sense if I adopt piqi_rpc_v1.piqi for
a "distributed erlang" type of transport?

I feel like I`m misunderstanding something here. Are we even supposed
to be reusing piqi_rpc_v1.piqi for implementing a different kind of
L7 type of transport, like distributed erlang like in the example
above?

Regards,
Motiejus

[1]: https://github.com/alavrik/piqi/blob/1e6089805b97986c6652d7a260b6645788a0e128/src/piqi_rpc_v1.piqi#L100-L106
[2]: https://github.com/alavrik/piqi/blob/1e6089805b97986c6652d7a260b6645788a0e128/src/piqi_rpc_v1.piqi#L19

[Kenneth Adam Miller]

Gosh, this is getting really complicated. Instead of creating our own format and all of that, can we just use gRPC, and add support for that in the proto files that are generated? gRPC is google's version of RPC that is a language bridge. It allows you to do declarative function exposure and just generate what you want automatically. It's meant to go hand in hand with protobuf. Otherwise, we're just reinventing the wheel.

Also, if we add support for this, that would be really cool because that would eliminate the headache of maintaining the erland RPC. You would have one specification, and it would work across languages. Not saying that you would have to do that, but obviously it reduces complexity in the final approach, and seems to be a natural choice for me.

--
You received this message because you are subscribed to a topic in the Google Groups "piqi" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/piqi/SR7Dd7yqC8E/unsubscribe.
To unsubscribe from this group and all its topics, send an email to piqi+uns...@googlegroups.com.

[Anton Lavrik]

gRPC definitely brings some new light to the conversation.

To be fair, piqi-rpc predates gRPC which was announced less than a month ago. Moreover, the thing we've been discussing in this thread is just a cleanup and generalization of the existing piqi-rpc system.

Both gRPC and piqi-rpc have declarative interface definitions, stubs generation and both go hand in hand with protobuf. It should be possible to provide interoperation between the two at definition and protocol levels. Can't see why not.

By the way, I've never said that pure OCaml implementation for piqi-rpc is impossible. Go for it if you need it. Going through Erlang is just a faster and more practical way to achieve this result if I were doing this.

Speaking of complexity, I can't see how https://github.com/alavrik/piqi/blob/rpc/src/piqi_rpc_v1.piqi is more complicated than https://github.com/grpc/grpc-common/blob/master/PROTOCOL-HTTP2.md Piqi-rpc looks an order of magnitude simpler to me.

Among other things, Piqi-rpc makes it extremely easy to make calls using JSON over HTTP1. It's proven to be really useful. gRPC doesn't have that.

The spec I suggested in this thread will make piqi-rpc transport-independent. gRPC is tied to HTTP2 (I think they made a mistake by not supporting HTTP1 from the beginning).

Overall, Piqi-RPC is a multi-transport and multi-format system that focuses on simplicity of interfaces and flexibility (especially on the server). gRPC is protobuf over HTTP2 and focuses on providing end-to-end tooling for client and servers including runtimes, stubs generation for both sides, auth support, out of the box compression, etc.

I'd say Piqi-RPC is much easier to implement and tweak. gRPC should be a great mainstream alternative, but more difficult to implement and less flexible. A great practical addition to Piq-RPC would be the ability to interact with gRPC clients.

Anton

--

You received this message because you are subscribed to the Google Groups "piqi" group.

To unsubscribe from this group and stop receiving emails from it, send an email to piqi+uns...@googlegroups.com.

[Kenneth Adam Miller]

Ok, now that you say that I can see what you mean and don't disagree. I've never used the rpc facilities for erlang, so when I said complicated, I wasn't talking about protocol or operation specs; I hadn't looked at either of what you have linked, so I was speaking more from unfamiliarity.

[Anton Lavrik]

Hi Motiejus,

This is actually pretty easy. Consider a general RPC scheme in which we have a client, piqi-rpc and a server.

So far, piqi-rpc implements this:

client ----(piqi-rpc/http)----> piqi-rpc/erlang ---> server/erlang

Kenneth has been asking whether we going to do the same thing for OCaml:

client ----(piqi-rpc/http)----> piqi-rpc/ocaml ----> server/ocaml

I suggested that we generalize piqi-rpc to work over arbitrary transports and then do this instead:

client ----(piqi-rpc/http)----> piqi-rpc/erlang ----(piqi_rpc_v1/pipe)---> piqi-rpc/ocaml ---> server/ocaml

In the above scheme, piqi-rpc/erlang acts as a proxy, taking piqi-rpc requests over http and sending them to ocaml worker over pipe and handling responses similarly but in the opposite direction. So, piqi_rpc_v1.piqi describes the protocol between piqi-rpc/erlang and piqi-rpc/ocaml over pipe in the above scheme. Think of it as SCGI.

This also explains why I added http-headers as a field to the piqi-rpc protocol header. This is so that server/ocaml could access http headers sent by the client. The same reason why somebody earlier asked to add access to Webmachine's wrq from piqi-rpc-based erlang servers. We could add transport level-features there as well, such as client-ip. In other words, these are just fields passed through by the proxy.

You can probably see now that the above scheme is very close to what piqi-rpc/erlang does today when it receives a JSON request and need to convert it to protobuf before giving it to the erlang server:

client ----(piqi-rpc/http)----> piqi-rpc/erlang ----(piqi_rpc_ad_hoc/pipe)---> "piqi server/ocaml"

                                           piqi_rpc/erlang <----(piqi_rpc_ad_hoc/pipe)<--- "piqi server/ocaml"

                                           piqi_rpc/erlang -----> server/erlang

What I am suggesting is to generalize piqi rpc protocol over any serial transport, such as pipe, sockets, zeromq, and so on. This list excludes HTTP because this protocol is already established and it is pretty good -- I don't think we need to change anything there.

If you want to use piqi-rpc over distributed erlang like this:

client/erlang ---(piqi_rpc/v1erl_dist) ---> piqi-rpc/erlang ---> server/erlang

I would model the representation based on piqi_rpc_v1 but come up with some native Erlang representation for the header and the body just to avoid costly Protobuf serialization in between. {ok|error, Response} may or may not be adequate enough depending on whether you want to proxy extra information.

Finally, a lot of good things will come out of this exercise:

- protocol-independent logic of piqi-rpc will move to piqi-erlang and piqi-rpc will become a general-purpose proxying project. this will also eliminate the need for a separate piqic-erlang-rpc, as protocol-independent server stubs will be generated by piqic-erlang.

- piqi-rpc will allow writing piqi-rpc services over pipe in any language without having them to implement an http server and http handling logic, request/response format conversion (piqi-rpc will happily do it for them) and so on -- they will need to have protobuf support, read from stdin and write to stdout.

- adding support for new transports to piqi-rpc will become much easier

- "piqi server" will have a standard and well define interface -- this allows to distribute "piqi" as a standalone package, and remove the explicit dependency on it in piqi-erlang. 

- "piqi server" itself can be seamlessly exposed as a piqi-rpc server over http by having piqi-rpc proxy http requests to "piqi server" over pipe. In other words, all "piqi" command-line tools will become automagically available as HTTP API. I think this would be pretty cool. Best of all, it will require 0 additional work.

- ocaml users will get a simple way to expose RPC services over HTTP -- something what piqi-rpc erlang users have been enjoying for years.

- all this will lay a solid foundation for adding piqi & piqi-rpc support to other programming languages.

Anton