Configuring number of worker threads spun by GRPC synchronous server C++

[Anirudh Kasturi]

Hello folks,

I noticed that when I bring up my grpc server, it spins up worker threads using the grpc::ThreadManager::WorkerThread::Run().

The number of worker threads spun = Number of Cores on the server I bring up my GRPC server on.

For eg : If I have a VM with 2 cores,  the grpc server brings up two worker threads. If the VM has 40 cores it brings up 40 worker threads.

In reality, I just need one worker thread. Is it possible to bring up only one worker thread?  Is that configurable? Also is there a way I can name the thread so that we have a way to track it?  Thank you.

Best,

Anirudh

[Sree Kuchibhotla]

Hi Anirudh,

There is no direct way of reducing the number of worker threads. Btw, I am assuming you are using gRPC version earlier than the latest 1.6. 

If so, when you are using a Synchronous grpc server, it by default creates as many "completion queues" as the number of cores (In the latest version of grpc, we only create ONE completion queue). 

Each completion queue will have "atleast one" worker thread polling for incoming new RPCs  (i.e If a worker thread is busy handling an RPC it just received, we create a new worker thread to poll for new RPCs. So clearly, on a server that handles multiple RPCs in parallel, the number of worker threads at any point may be greater than 1).

In anycase, you can reduce the "minimum number of workers" to 1 by having the Sync server create only one completion queue the following way:

ServerBuilder builder

...

builder.SetSyncServerOption(ServerBuilder::SyncServerOption::NUM_CQS, 1)

(Here is an example of a grpc test  that sets the number of completion queues to 4)

thanks,
Sree

--
You received this message because you are subscribed to the Google Groups "grpc.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email to grpc-io+unsubscribe@googlegroups.com.
To post to this group, send email to grp...@googlegroups.com.
Visit this group at https://groups.google.com/group/grpc-io.
To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/c744a0c7-c248-4a16-9bf1-df7fba045fe0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Anirudh Kasturi]

Thanks a lot Sree.  This is very helpful.  I will also make sure I upgrade my grpc version.

Best,

Anirudh

[Anirudh Kasturi]

Hi Sree it worked.  So one more question.  I believe there is a timeout for the worker threads.  In case there are more requests coming in and the grpc server spins up new worker threads from the pool based on the incoming requests,  once those requests are processed,  would the grpc server spin them down back again?  Is it that elastic?  

Thanks,

Anirudh

[Sree Kuchibhotla]

Yes, it is elastic. 

It will dial the number of threads down to 1 thread  (to be more precise, it will dial down to whatever is set in the ServerBuilder::SyncServerOption::MIN_POLLERS setting - which is "1" by default. Also, this is a "per-completion" queue setting)

thanks,

-Sree

[Anirudh Kasturi]

Great !  I found on github for grpc version 1.6.0, the default settings are :

num_cqs(1), min_pollers(1), max_pollers(2), cq_timeout_msec(10000) {} Probably I am better off going with the latest version then. Thanks again Sree.

[Sree Kuchibhotla]

Hi Anirudh,

Sure you could go with latest version - but keep in mind that we MAY change num_cqs() to be equal to the number of cores in a future version. Same goes for other settings - but they are less likely to change though.

So if having one completion queue and a minimum of one polling thread is a hard-requirement for you, I recommend actually overriding the defaults in your code (like I mentioned earlier)

-Sree

[Anirudh Kasturi]

Hi Sree,

Thanks for the heads up.  

Best,

Anirudh

[soory...@gmail.com]

Hi all,

   I have a use case , where Im building the grpc server as a shared  library and it exposes on runServer(const std::string& server_address) api to application.

Application and link to the server library and call run server with desired ip:port, server will start

But the problem is, When apllication calls this runServer function, builder.BuildAndStart() fails with "At least one of the completion queues must be frequently polled"(server_builder.cc:341 - grpc_v1.20 )

    ServerBuilder builder;

    ......

    m_server = builder.BuildAndStart();

But if I compile the application source code along with the library and use, no issues are coming.

Can anyone shed some light on how to proceed?

Thanks and Regards,

A Sooryaa

[Navys & GCG]

did you fix this problem? I am facing the same issue.

вторник, 9 июля 2019 г. в 12:56:40 UTC+3, soory...@gmail.com: