callback API and details on threading model

Timo

unread,

Sep 18, 2022, 9:29:16 AM9/18/22

to grpc.io

I did research on this topic but did not find detailed information in the documentation yet.

How exactly does the thread model of the new callback API work?

When using the synchronous API, the thread model I guess is this:

- grpc owns threads, number can be limited

- Several RPCs can operate on one thread, but there's a limit

- When too many RPCs are open, the client receives a "resource exhausted"

- An application with multiple clients needs at least one thread per each open RPC.

In the callback (not asynchronous) API, I understand:

- grpc owns threads and spawns new threads if needed

- multiple RPCs can be handled on one thread non-blocking

For the server, I wonder how this scales with many (don't have a number in mind) RPCs being open. Assuming all 16 threads are spawned, how many RPCs can I operate?

Assuming I have an application with multiple clients implemented, each connecting to different servers.
Would all the clients be able to share the same thread pool, or would (in worst case) each client spawn 16 threads?

Especially when designing microservices where each service offers a server, but can be a client to another service it may be important to not scale threads too much.

Thanks

Zach Reyes

unread,

Sep 21, 2022, 4:54:19 PM9/21/22

to grpc.io

What language of gRPC? That'll allow me to route it to the correct person to answer.

Naman Shah

unread,

Sep 25, 2022, 11:31:14 PM9/25/22

to grpc.io

Hey Zach, I have the same question about the implementation in CPP.

Timo

unread,

Mar 23, 2023, 8:12:49 AM3/23/23

to grpc.io

Hey Zach, I thought I answered this, but seems I missed. The question is about C++.

AJ Heller

unread,

Mar 29, 2023, 2:16:08 PM3/29/23

to grpc.io

Sometime this year, gRPC C++ will switch to by default have a single auto-scaling thread pool per process, and all of gRPC's threaded activities will utilize it. Applications will have some control over this, though, by being able to provide custom EventEngine instances per channel or per server. See https://github.com/grpc/grpc/blob/ec1d75bb0a24a626e669696bb48490e7ac40cc69/include/grpc/event_engine/event_engine.h

The question of "how many RPCs can I operate" will depend heavily on how much work your services are doing, and maybe how quickly the pool needs to scale.