What is difference between call and notification completion queues?

[afshi...@gmail.com]

I'm learning gRPC and when I was checking asynchronous servers, I noticed that a call for obtaining gRPC requests like:

"service_->RequestSayHello(&ctx_, &request_, &responder_, new_call_cq_, notification_cq_, this);"

takes 2 completion queues as input. From their name, it seems that they are for finding getting new calls and for notifications separately. I have seen that same queue is used as for both, but is there any benefit for using 2 different queues for each one?

In addition, what factors are important for selecting number of completion queues? Should I select 1 per service or 1 per rpc call? or use other factors such as selecting 1 for a highly accessed service?

[Vijay Pai]

The new call CQ is for the initial tag identifying the new call's initiation. The call CQ is for further operations on that call (like Read/Write/Finish). You can use the same CQ for both and that is probably the most common/recommended configuration.

Factors for selecting CQ count: in almost all cases, you should use 1 CQ for the entire server. This is the best for efficiency and avoiding work stranding, as long as your CQ processing thread is able to process any type of tag that comes back. Some of our performance benchmarks use a separate CQ for each CQ processing thread just to avoid CQ contention in the extreme cases, but that's not a major factor except in a very small number of cases. Generally the advice is to use 1 CQ for the whole server.

[afshi...@gmail.com]

Thanks for the info.

After I asked this question, I found this about performance trade-offs that said best is to use 1 thread per cpu core and 1 CQ per thread. But if you say normally 1 CQ is ok for whole server, I will go with this idea.

Best Regards