How gRPC Server Completion Queue manage its connections

[Jingsong Meng]

Hello Guys.

A quick question is that: If i use one completion queue in server side to handle all of the connections, for each return from cq.AsyncNext, how do i know which connection it is.

Explanation: based on https://github.com/grpc/grpc/blob/master/doc/core/epoll-polling-engine.md, I know one completion queue has one pollset. In server side, i can open 1 CQ and it could handle multiple connections. In greeter_async_server.cc (helloworld example), server process each rpc service by calling cq->Next(&tag, &ok), then static_cast tag into CallData, and call ServerAsyncResponseWriter.Finish to reply data to client. cq->Next will call AsyncNextInternal, inside which, it will call grpc_completion_queue_next and FinalizeResult. FinalizeResult will deserailize message from connections. So I think ServerAsyncResponseWriter.Finish is used to send message, and FinalizeResult is used to receive message. But how do I know where the message come from or send to if I use one completion queue in server side?

Thanks in advance for all the insights.

[Christopher Warrington - MSFT]

On Friday, November 19, 2021 at 3:07:02 PM UTC-8 meng...@gmail.com wrote:

> If i use one completion queue in server side to handle all of the
> connections, for each return from cq.AsyncNext, how do i know which
> connection it is.

You will need to augment CallData with these sort of details. You will also
need multiple instances of CallData at once, one per "in flight"
gRPC operation.

In an older thread on a similar topic, "How to implement async gRPC with
more than one method?", [1] I provided an architectural sketch of one way to
make this work. See if that helps.

[1]: https://groups.google.com/g/grpc-io/c/bXMmfah57h4/m/EUg2B8o1AgAJ

--
Christopher Warrington
Microsoft Corp.

[Jingsong Meng]

Thanks. I'm learning your thread  https://groups.google.com/g/grpc-io/c/bXMmfah57h4/m/EUg2B8o1AgAJ. 

And i think server handle all new connections in 'on_read' function (inside src/core/lib/iomgr/tcp_server_posix.cc). For each connection, server create a fd for it (so one connection bind with one fd), and inside pollset_work (src/core/lib/iomgr/ev_epollex_linux.cc), the pollset of the CQ will epoll_wait all of the fds. I think I have to pass the (connection, fd) pairs to CQ. Right now, the problem is that inside Finsh and FinalizeResult, I only have 'tag' (I guess due to some encapsulation reasons, grpc hide low level information here), I didn't have any hint to locate which connection/fd from this 'tag'. I think grpc have some record to pair this 'tag' with connection. And if one client send multiple requests, each request has its own 'tag', but grpc will find the connection/fd from these tags. So I think I have to find this 'tag' - connection map inside grpc. But not find yet. Please correct me if i'm wrong.

Thanks for any ideas or suggestions. 

[Christopher Warrington - MSFT]

On Monday, November 22, 2021 at 7:09:22 PM UTC-8 meng...@gmail.com wrote:

> I didn't have any hint to locate which connection/fd from this 'tag'. I
> think grpc have some record to pair this 'tag' with connection. And if one
> client send multiple requests, each request has its own 'tag', but grpc
> will find the connection/fd from these tags. So I think I have to find
> this 'tag' - connection map inside grpc. But not find yet. Please correct
> me if i'm wrong.

Ah. You're asking about things like "which client is on the other end of
this call". Look at ServerContext::auth_context [1] and ServerContext::peer
[2]. My memory of exactly how Finish and FinializeResult work together is
hazy. If you no longer have access to the ServerContext when
Finish/FinializeResult is running, you'll need to copy the value you need
from the ServerContext into whatever per-tag data structure you use.

Consider, though, whether you need to be tracking the connection on a "per
socket"/"per fd" level or if you need to be tracking a "connection" at a
higher level, like a single user's "session" in your service. What should
the behavior be in the client device changes networks, like switching from a
cell connection to a WiFi connection? Are those different "connections" or
the same?

[1]: https://grpc.github.io/grpc/cpp/classgrpc_1_1_server_context.html#a2b28535186cbe2d7b6ed1ff59980191b
[2]: https://grpc.github.io/grpc/cpp/classgrpc_1_1_server_context.html#a6e94cf9e11550b34ab98271202f03adc