Server Threadpool Exhausted

[Roshan Chaudhari]

i have gRPC sync server with one service and 1 RPC.

I am not setting ResourceQuota on serverbuilder. If n clients wants to connect, there will be n request handler threads created by gRPC. I want to keep some limit on these threads. lets say 10. And if it costs some latency in serving client, it is okay.

So I tried these settings:

grpc::ServerBuilder builder; grpc::ResourceQuota rq; rq.SetMaxThreads(10); builder.SetResourceQuota(rq); builder.SetSyncServerOption( grpc::ServerBuilder::SyncServerOption::MIN_POLLERS, 1); builder.SetSyncServerOption( grpc::ServerBuilder::SyncServerOption::MAX_POLLERS, 1); builder.SetSyncServerOption(grpc::ServerBuilder::SyncServerOption::NUM_CQS, 1);

From another process, I am firing up 800 clients in parallel. So I expect there will be 1 completion queue for each of them and 10 threads sharing it. However, on client side there is an error:

"Server Threadpool Exhausted"

and none of the client succeeds.

[Roshan Chaudhari]

More context:
I am using C++ sync server. Currently when I have multiple concurrent  clients, number of threads used by the server increases linearly and it seems each client is served with separate thread.

streaming RPC I am using, will be idle 90 percent of the time, so rarely data will be sent across it. So server can have minimal number of threads and multiple client requests can be served by say fixed number of threads. And it is okay if there is some delay in serving the client.

Is it possible to achieve this in sync server? Or async is the only option.

[yas...@google.com]

Hi, when wanting such granular control over the threading model, it's better to use the async model. Currently, the CQ based async API is the only API that can serve this purpose. I would've wanted to recommend using the callback API along with the ability to set your own `EventEngine`, but we don't have that built out yet.