gRPC unary calls performance and client-side implementation for bi-directional streaming

[Preetham Kalagara]

I'm trying to solve the following problem using grpc. we have one ml service which returns positivity score given a text.
On the client side, we will receive multiple texts parallely and we need to get the score from ml service for each of those texts
I implemented that using unary calls with python on grpc server side and java on client side.

Couple of questions on this,

• I see in docs that we can share the same channel across multiple stubs/client objects. how to know the maximum requests a channel can handle?
• how to get to know the time taken at each step for every grpc call? like time taken to create connection, time taken to send the request and so on. Any tool I can use to track things like these?
• Can I look into bi-directional streaming instead of using unary calls with multiple clients and channels?

And one more basic question, how to implement request-response kind of model in bi-directional streaming. I'm looking only for the basic idea of client side implementation using grpc-java. How do we get to know the response of a particular request in bi-directional streaming without passing some identifier for every request and mapping the response through that id?

[Amit Saha]

On Friday, March 18, 2022 at 2:01:02 PM UTC+11 Preetham Kalagara wrote:

I'm trying to solve the following problem using grpc. we have one ml service which returns positivity score given a text.
On the client side, we will receive multiple texts parallely and we need to get the score from ml service for each of those texts
I implemented that using unary calls with python on grpc server side and java on client side.

Couple of questions on this,

• I see in docs that we can share the same channel across multiple stubs/client objects. how to know the maximum requests a channel can handle?

I recommend reading https://grpc.io/docs/guides/performance/ and the linked issue: https://github.com/grpc/grpc/issues/21386

• how to get to know the time taken at each step for every grpc call? like time taken to create connection, time taken to send the request and so on. Any tool I can use to track things like these?

For a RPC call, a connection is not setup, unless of course it was disconnected and had to be re-established. https://grpc.io/blog/a-short-introduction-to-channelz/ may be useful for you. You can of course publish your own metrics using interceptors/middleware in your client and server.

• Can I look into bi-directional streaming instead of using unary calls with multiple clients and channels?

It depends on what you are trying to solve. Streaming has some limitations such as poor load balancing, for example. If Unary RPCs work for your use-case, might be worth keep using it? If you search on the Internets for streaming RPCs, that's what folks seem to be suggesting - that is, you should evaluate your use-case first and then decide whether you need bi-directional streaming. 

And one more basic question, how to implement request-response kind of model in bi-directional streaming. I'm looking only for the basic idea of client side implementation using grpc-java. How do we get to know the response of a particular request in bi-directional streaming without passing some identifier for every request and mapping the response through that id?

gRPC guarantees the order of delivery of messages in a streaming call, however, as you rightly guess, if a message takes longer to be processed, the reply may arrive after a reply to a later message. So, I think you will need to implement your own logic to guarantee - something that you have thought about already.