Bidi streaming benefits

[Jeroen Meijer]

Hi,

I am building a messaging system that should handle several thousands of messages per second. I am using Java and gRPC calls to relay messages from clients to servers.

Currently, I am performing a unary non-blocking RPC per message. I wonder however, what would be the expected performance benefit of implementing this in a bidi streaming way? Meaning that instead of performing an RPC for each message, each message is relayed as a message in an already in open streaming RPC. Implementing this would be quite some effort, this is why I am asking about your experience and whether this might be worth it.

The way I would approach this is have a dynamic object pool of open bidi streaming calls. The size of the pool would be the allowed concurrency limit determined client side. Furthermore, I may enhance every message in the stream with e.g., standard gRPC headers, trailers and status codes to indicate result of relaying the message. Is there a standard and recommended approach for this?

As an aside, in case of unary RPCs, what is really the benefit of reusing stubs, does that improve performance in a meaningful way?

I am interested in your opinion and experience.