Building a grpc reverse proxy using client headers

[chetan...@gmail.com]

Hi

I am planning to build a grpc-reverse proxy that sits in fronts of a few of our grpc services.

Requirement:

We have a client A that at the moment talks to B or C (based on internal logic). We want to move this logic to the proxy layer.

So we will introduce a new grpc-reverse-proxy server P sitting in front of B and C and redirect requests based on the headers passed and the application logic. Below is how I'm thinking of designing it-

A  (starts calls to P with custom headers )

   -> P

        - Implements ServerCallHandler to read headers and run application logic

        - Has 2 channel objects that it uses to connect to B & C as a client

        - Based on the logic proxies the request to either B or C

   -> B or C

        - Processes the request and returns the response back to P

   -> P

       - Proxies the response back to A 

I have leveraged some of the code from here - https://github.com/ejona86/grpc-java/commit/29728aeb003ced3c190197c176563643be22bef1, to build the logic described above and it works.

I couldn't find a lot of documentation on how the client and server in gRPC internally talk to each other and what are the best practices when writing your own ServerCallHandler, ClientCall.Listener and ServerCall.Listener; particularly a sequence diagram of how startCall(), onMessage(), onHalfClose(), onCancel() and onReady() are called and what are the guarantees gRPC provides. 

I would like to know if my approach to tackle the requirement sounds acceptable and whether there are things that I should take into consideration when building this solution. One thing for example is that the clients will have to create a new stub for each request to add custom headers before starting a call. Is this good or bad? I don't see any problem with it in terms of how gRPC works but would like to know if it will break something I'm not aware about.

Thanks

Chetan 

[sanjay...@google.com]

Have you considered using the Envoy proxy with static configuration so you don't need to develop a reverse proxy of your own?

[chetan...@gmail.com]

I did look at Envoy proxy. The business logic to redirect traffic is dynamic and involves fetching latest information from another distributed system and I couldn't find a straight forward way to do it using Envoy proxy. 

My requirement to direct traffic is not static or deterministic, hence building a custom solution seems a better option.

[sanjay...@google.com]

Envoy proxy uses xDS to dynamically change the routing configuration - some of these can be header matching rules that encapsulate your dynamic logic. This of course requires an "xDS" server to supply the route (and other dependent) configuration. 

In gRPC too there is support for xDS on the client side and very soon there will be support for header based routing as described here which means the client itself does the header based routing. But this too requires putting your logic in an xDS server similar to above (once the feature is available in gRPC Java).

[chetan...@gmail.com]

Hi. Thanks for sharing the information regarding Envoy proxy. I think it will require a lot of changes for me to integrate Envoy proxy in our architecture and writing a simple reverse proxy (if it works) the way I described above is an easier, faster and better option. Do you have any feedback on how it can be achieved directly through implementation on gRPC interfaces?

[sanjay...@google.com]

> how it can be achieved directly through implementation on gRPC interfaces?

Nothing offhand that comes to mind. Probably the original code you leveraged is a good starting point.

[Eric Anderson]

Using a gRPC-based proxy to provide your own logic sounds good and fine. Since you just need headers you could make your own HTTP/2 proxy, but there's nothing wrong with using gRPC as the proxy in my mind.

On Thu, Aug 6, 2020 at 1:47 PM <chetan...@gmail.com> wrote:

I couldn't find a lot of documentation on how the client and server in gRPC internally talk to each other and what are the best practices when writing your own ServerCallHandler, ClientCall.Listener and ServerCall.Listener; particularly a sequence diagram of how startCall(), onMessage(), onHalfClose(), onCancel() and onReady() are called and what are the guarantees gRPC provides.

All of that documentation should be in the respective class's documentation. You probably want to look at the ClientCall and ServerCall classes first. The ClientCall.Listener sort of mirrors ServerCall and the ServerCall.Listener mirrors ClinetCall.

You may find https://github.com/grpc/grpc/pull/15460 helpful. It is basically the documentation that already exists in Java, but reworded a bit. It isn't merged because of nomenclature reasons; it is pretty accurate, just the precise names we use were in debate.

I would like to know if my approach to tackle the requirement sounds acceptable and whether there are things that I should take into consideration when building this solution. One thing for example is that the clients will have to create a new stub for each request to add custom headers before starting a call. Is this good or bad? I don't see any problem with it in terms of how gRPC works but would like to know if it will break something I'm not aware about.

It's not bad at all. I assume you're saying you'd use stub.withInterceptors() which would return a new stub. That's no issue. Stubs are cheap.

You do have another option though: you could have an interceptor that will manage the header always present in the stub and provide it configuration via stub.withOption(). withOption() also creates a stub. The only difference is that you can expose a higher-level API via the CallOptions.Key instead of directly modifying the Metadata at each call site. This doesn't matter if there's only a few call sites sending RPCs. But if you end up having many you may find it advantageous.