What's the Threading Model behind Completion Queue?

[Lixin Wei]

I'm wondering what's the threading model behind the completion queue?

Who produces items to the completion queue? What is between the completion queue and the network?

[AJ Heller]

Hi Lixin. Good questions! I can offer a high-level summary.

> I'm wondering what's the threading model behind the completion queue?

This is a bit of an oversimplification, but the C++ API's `CompletionQueue` borrows threads from the application. Work is done when applications make a blocking call to `CompletionQueue::Next`. See the API docs here https://grpc.github.io/grpc/cpp/classgrpc_1_1_completion_queue.html#a86d9810ced694e50f7987ac90b9f8c1a.

> Who produces items to the completion queue?

Applications do, for the most part. Some of this is covered in the C++ Asynchronous-API tutorial: https://grpc.io/docs/languages/cpp/async/. 

> What is between the completion queue and the network?

Quite a few things - the majority of gRPC sits between them. At a high level, there's the transport layer, handling things like HTTP/2 and cronet transports. Then there are filters that both filter and augment calls, adding things like max_age filtering and load balancing for client channels. The bottom-most layer is called iomgr, providing things like network connectivity and timers.