Server side transport keepalive in C++

[deepako...@gmail.com]

Hi,

I am using gRPC C++ async server for single/unary RPC. One of the problems I noticed is that if a client disappears after making successful RPC call(s) i.e. client doesn't gracefully shutdown the transport, server keeps the transport  open forever.  I am looking at the TCP connection on server and it remains open forever. Since gRPC library doesn't expose underlying transport to the application, is there any knob that can allow server to close such stale TCP connections?

I did try to use GRPC_ARG_KEEPALIVE_TIME_MS but no luck. I tried changing the greeter_async_server.cc in example/cpp/helloworld as follows:

    ServerBuilder builder;

    // Listen on the given address without any authentication mechanism.

    builder.AddListeningPort(server_address, grpc::InsecureServerCredentials()/*grpc::SslServerCredentials(sslCredentialsOptions)*/);

    // Register "service_" as the instance through which we'll communicate with

    // clients. In this case it corresponds to an *asynchronous* service.

    builder.RegisterService(&service_);

    // Get hold of the completion queue used for the asynchronous communication

    // with the gRPC runtime.

    cq_ = builder.AddCompletionQueue();

    builder.AddChannelArgument(GRPC_ARG_KEEPALIVE_TIME_MS, 20000);                            <====

    builder.AddChannelArgument(GRPC_ARG_KEEPALIVE_TIMEOUT_MS, 10000);                    <====

    builder.AddChannelArgument(GRPC_ARG_HTTP2_BDP_PROBE, 1);                                      <====

    //builder.AddChannelArgument(GRPC_ARG_KEEPALIVE_PERMIT_WITHOUT_CALLS, 1);

    // Finally assemble the server.

    server_ = builder.BuildAndStart();

    std::cout << "Server listening on " << server_address << std::endl;

Test Run:

========

bash-4.2$ ./greeter_async_server &

[1] 9592

bash-4.2$ Server listening on 0.0.0.0:50051

TCP:

-------

bash-4.2$ ss -antp | grep async

LISTEN     0      128         :::50051                   :::*                   users:(("greeter_async_s",pid=9592,fd=4))

bash-4.2$

RPC call from client:

---------------------------

bash-4.2$ Server listening on 0.0.0.0:50051

Processing call

Client disappeared but transport remains open forever:

-------------------------------------------------------------------------

bash-4.2$ ss -antp | grep async

LISTEN     0      128         :::50051                   :::*                   users:(("greeter_async_s",pid=9592,fd=4))

ESTAB      0      0         ::ffff:172.23.137.19:50051              ::ffff:172.23.142.2:59686               users:(("greeter_async_s",pid=9592,fd=7))

bash-4.2$ ss -antp | grep async

LISTEN     0      128         :::50051                   :::*                   users:(("greeter_async_s",pid=9592,fd=4))

ESTAB      0      0         ::ffff:172.23.137.19:50051              ::ffff:172.23.142.2:59686               users:(("greeter_async_s",pid=9592,fd=7))

bash-4.2$

Regards,

Deepak

[Arpit Baldeva]

What you need to use is the GRPC_ARG_MAX_CONNECTION_IDLE_MS option. However, that option is currently buggy. Before 1.9.0, it could cause a crash and starting 1.9.0, it could cause memory leak if enabled. See this issue - https://github.com/grpc/grpc/pull/13594