grpc server becomes unusable after client crash

81 views
Skip to first unread message

Dinesh

unread,
Aug 18, 2016, 6:41:38 AM8/18/16
to grpc.io
Hi, I am using grpc v0.13, async client and server. If client crashes while sending requests to server, then server is not able to process any further requests. Even if I restart the client, a new process, then all the requests fail with the below error.
error_code= [13], details= Received RST_STREAM err=2.
This error goes away only when the server is restarted.

When client crashes then cq->Next gives 2-3 requests with "!ok". Do we need to handle it somehow/do something in this case ?

Thanks,
Dinesh

----- Code snippet of server ---------------

  while (true) {
    void* tag;
    bool ok;

    if (!cq_->Next(&tag, &ok)) {
      // Queue is shutting down.
      break;
    }

    if (!ok) {
      // Non regular event, skip.
      continue;
    }

    function<void()> handle_rpc_cb = [tag]() {
      ServerRpcBaseContext* ctx = ServerRpcBaseContext::detag(tag);
      ctx->Proceed();
    };

Nathaniel Manista

unread,
Aug 18, 2016, 3:06:20 PM8/18/16
to Dinesh, grpc.io
On Thu, Aug 18, 2016 at 3:41 AM, Dinesh <dinesh...@gmail.com> wrote:
Hi, I am using grpc v0.13,

What is it that has you using such an old version of gRPC? Is there any chance you can upgrade to 1.0.0-pre2 or later?
-Nathaniel

din...@cohesity.com

unread,
Aug 21, 2016, 2:28:16 PM8/21/16
to grpc.io, dinesh...@gmail.com
HI Nathaniel, Now I am using latest grpc code from git using git clone. I am running grpc_async_server and greeter_async_client2. I used 10000 loops in async_client2. This issue happens with large payload. I used 100kb request size. Steps to reproduce:

Run grpc_async_server
Run greeter_async_client2 with 10k loops, each request payload 100kb.
Kill client in between.
Run client again. Server stops responding. Issue can be resolved only by restarting server.

Issue does not happen if request size is very small, like 10-20 bytes.

din...@cohesity.com

unread,
Aug 21, 2016, 2:32:13 PM8/21/16
to grpc.io, dinesh...@gmail.com, din...@cohesity.com
I am using greeter_async_server and greeter_async_client2 provided in examples/cpp/helloworld.
Reply all
Reply to author
Forward
0 new messages