Load balancing and metrics

[shinzui]

Hi, 

I just started evaluating Cap'n Proto as an alternative to gRPC in a microservice architecture, and I am having a hard time finding material on deploying services that are using Cap'n Proto RPC. Is anyone using Cap'n Proto RPC with kubernetes and a service mesh? How do you load balance your services? How do you capture metrics? Is the protocol suitable for cloud-native apps? 

My naive first look makes me think that it's not suitable for a microservice architecture with a lot of stateless services. I would appreciate any pointers to prove me wrong. 

Thank you.

[Kenton Varda]

Hi,

While Cap'n Proto is certainly capable of replacing gRPC here, obviously Google has a lot more people working on the gRPC ecosystem, and so more infrastructure has been built out there. With Cap'n Proto you will have to do more things yourself. If your needs fit well into the model supported by common gRPC infrastructure, this may not be worth it to you. On the other hand, if you have a more unusual use case, then you might find you have to build custom solutions either way, in which case Cap'n Proto's more fundamental benefits (serialization performance and object capabilities) may make it a better choice.

Cap'n Proto is especially powerful for:

* Stateful services, where nodes across a cluster need to control and hold on to the state of other nodes. For example, Sandstorm.io (startup I founded) built a cluster-scaleable version of Sandstorm called "Blackrock" which is itself a container orchestrater designed to run many heterogeneous container instances on behalf of individual end users. This is a fundamentally stateful thing, since each container is serving a specific user with specific state and can't simply be interchanged with others. Using Cap'n Proto as the underlying communications protocol made this a lot easier to manage.

* IPC use cases, where services are running on the same machine and can communicate via unix sockets or even shared memory. Cap'n Proto's CPU performance shines here (while its somewhat higher bandwidth usage becomes irrelevant). Running multiple services on the same machine isn't really in style right now, but my current employer, Cloudflare, does a lot of it -- every one of the machines in our edge network runs an identical set of services so that any one machine can handle any kind of request on its own.

But if you're doing a standard kubernetes microservices thing... probably going with gRPC is going to be a lot easier right now.

-Kenton

--
You received this message because you are subscribed to the Google Groups "Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email to capnproto+unsubscribe@googlegroups.com.
Visit this group at https://groups.google.com/group/capnproto.

[Kevin Conway]

I apologize for bumping an old thread. I'm curious if your recommendation has changed any since 2018.

I have several projects that match the IPC and stateful use cases you mention that I'm going to try building on capnproto as a learning experience. I'm also interested, though, in an option to have only one service definition and schema language, one code generation toolset, etc. The only hesitance I have in replacing gRPC with capnproto for the stateless, kubernetes deployed, auto-scaled type of system is this old thread which seems to be one of the few that mentions this subject.

My current understanding of how pipelining is implement is that it only works for returned interfaces. That is, an interface method defined as `foo @1 (bar :String) -> (baz :String)` would not present me the option of pipelining that value through subsequent operations. If I wanted to "opt-in" to pipelining support then I would need to convert the return value to an interface type like what is done in the calculator example of the rust RPC implementation. If that's the case then that means I can support less stateful interactions by only ever defining methods that return concrete types as a way of forcing the RPC implementation to avoid pipelining. This is equivalent to only defining unary methods in gRPC or using one of the streaming signatures to "opt-in" to a stateful interaction. I'm new to capnproto so this description is fishing for clarification or disagreement.

Another way to talk about this might be to ask "what would be required to support a stateless subset of capnproto even if it meant losing many of the advantages of the RPC model?". If it comes down to "use a subset of the schema language (e.g. only define unary APIs) and implement a protocol aware reverse proxy" then that's a fairly accessible amount of work for a contributor to take on and document. After that, the other concerns like metrics gathering or integration of growing open standards is a matter of building code generation plugins. I'm curious to hear thoughts from folks with more capnproto experience.

To unsubscribe from this group and stop receiving emails from it, send an email to capnproto+...@googlegroups.com.

[Kenton Varda]

Hi Kevin,

You are correct that if you avoid passing interface types in your RPC messages, you will never have the opportunity to take advantage of promise pipelining, at least as the protocol is defined today.

But I'm not sure I understand the motivation for your question. Why do you want a "stateless subset of capnproto"? Is this about looking for an easier first step to implementing Cap'n Proto RPC in more languages? If so, I think what you're looking for is what we call "Level 0 support" in rpc.capnp. A level 0 implementation does not support passing interface references.

-Kenton

To view this discussion on the web visit https://groups.google.com/d/msgid/capnproto/403ddef7-c48e-4c6a-8751-60e84d34d3efn%40googlegroups.com.

[Kevin Conway]

I appreciate the confirmation.

> Why do you want a "stateless subset of capnproto"?

It's less about implementing the RPC protocol and more about an outlet for using familiar deployment and operational practices to get decent work distribution among mundane, 12-factor style systems. For example, if I have an interface defined that offers CRUD style operations for one or a few different structures then I don't really need the stateful portions of the capnproto RPC protocol and could potentially benefit from avoiding them. Keeping within the "stateless subset" would make it much easier to deploy/operate that system using an auto-scaling policy and protocol aware reverse proxy to distribute the requests just like most stateless style HTTP services that sit behind a load balancer. I don't see a technical barrier to that protocol aware proxy acting similar to a membrane by ensuring that all interfaces returned from interfaces, recursively, are bound to the same underlying instance of the system which would provide some support for more stateful transactions or "sticky binding" where it makes sense and could provide support for having stateful and stateless support side-by-side. If I had a large set of interfaces that all stayed within the "stateless subset" then I could potentially build an adapter that converts RPC calls to HTTP calls using a standard generation of paths, headers, and content bodies (similar to what Twirp does for protobuf). I'd be losing basically all of the RPC advantages that capnproto has but I'd be able to then buy into a fairly extensive suite of HTTP related tools for operations, telemetry collection, access/audit logging, etc.

To be clear, I'm learning capnproto because I have some pet projects that fit well into the expected use cases of stateful/distributed systems and IPC. The "stateless subset" line of thinking is trying to tease out if there's a reasonable path to replace all my current protobuf/gRPC use cases. It would be convenient to have only one design language and ecosystem to stay up to date on and contribute to. It would be a compelling point, to me, if I could say that capnproto + RPC covers all my typical cases and also has excellent optimizations for my complex or unique projects.

--

Kevin Conway

[Kenton Varda]

Yes, it's straightforward to implement a load-balancing proxy server that redirects each call to a different back-end. To do this in C++, you'd implement capnp::Capability::Server, implementing the generic `dispatchCall()` method such that it chooses a back-end and then uses `context.tailCall()` to forward the call. You'd use that as your "bootstrap" interface. There's no need for the proxy to know the schema of this interface; as long as all the back-ends that it talks to use the same bootstrap schema, any calls will flow through nicely.k

In fact, a proxy implemented this way would "just work" with calls that return capabilities! Any returned capabilities would automatically become "sticky", such that calls to them would forward to the specific back-end that originally returned them. The proxy itself remembers this forwarding on a per-connection basis; when the client disconnects, the forwarding tables are discarded. All this logic is pretty much built into the current implementation.

Heh, if I had more time I'd work on building out a bunch of these little tools, like a load-balancing proxy, as part of the core project...

-Kenton