Should I use async API instead of sync one ?

[Théo Depalle]

Hi everyone,

I just implemented a cpp interface for my software using synchronous gRPC API. Both clients and server will run locally.

I have multiple services with different kind of rpc :
- simple RPC to get/set value (fast computation time)
- server side streaming RPC to subscribe to continuous data informations

The server side rpc are "infinite" as the server sends data at a given frequency until the client disconnects. To not only have this kind of request in my thread pool and block simple rpc requests, I set a maximum client number for each streaming rpc.
Finally, I set the number of thread in my thread pool in order to have at least one thread for each request if I have :
- as many requests as the maximum number of clients for each streaming rpc
- 2 requests for each simple rpc.

Exemple : Let's consider a server with only one service containing 1 server side streaming rpc and 2 simple rpc. The maximum client number for the streaming rpc is 3. I will set my thread pool to 3 + 2 * 2 = 7 threads.

In normal use case I will have 2-3 clients maximum for each rpc but as I have a total of 18 simple rpc and 5 server side streaming rpc I can have a thread pool between approximately 50 and 70 threads. If my interface evolves I fear that I will have too many threads in my thread pool.

Do you think about other limitations I could have using the synchronous API ? Do you think it is inevitable to move to the async API for this kind of interface ?

Looking forward to your feedback!

Théo

[AJ Heller]

Hi Théo. You'll at least want to consider using the async API. See https://grpc.io/docs/guides/performance/#c for some handy guidelines. The sync API may be fine for fast, non-blocking server-side operations. For the streaming method, you've limited your concurrency there a bit, but that's not a big deal if you're only ever serving 3 clients concurrently. You might run into diminishing returns with such a high thread count, paying thread-switch costs for a large number of threads when your service is under heavy load. As to whether it's inevitable that you move to the async API ... if you have a working solution that performs well enough for your needs, and if you know it doesn't need to grow that much, then it may be reasonable to keep what you have. It's a judgement call, maybe something to benchmark and measure for yourself.