Backpressure with grpc c++.

[Frédéric De Jaeger]

Hi

Sorry for the noob question, I've just started grpc.  I'm a bit puzzled on how the backpressure can be done on the client side.

For instance, consider the canonical example `greeter_async_client2.cc`:

    for (int i = 0; i < 100; i++) {

       std::string user("world " + std::to_string(i));

       greeter.SayHello(user);  // The actual RPC call!

   }

This enqueues 100  requests to be sent (and everything is processed asynchronously, fine).

Suppose the server/network is slow and the client wants to enqueue faster than what the server can process, what will happen ?

My understanding of the sample code is that none of the API in `SayHello` blocks the caller:

        // stub_->PrepareAsyncSayHello() creates an RPC object, returning

       // an instance to store in "call" but does not actually start the RPC

       // Because we are using the asynchronous API, we need to hold on to

       // the "call" instance in order to get updates on the ongoing RPC.

       call->response_reader =

           stub_->PrepareAsyncSayHello(&call->context, request, &cq_);

        // StartCall initiates the RPC call

       call->response_reader->StartCall();

        // Request that, upon completion of the RPC, "reply" be updated with the

       // server's response; "status" with the indication of whether the operation

       // was successful. Tag the request with the memory address of the call object.

       call->response_reader->Finish(&call->reply, &call->status, (void*)call);

Request will then accumulate somewhere in a hidden queue (bad), or some request will be dropped (also bad), or one of these API actually blocks (still bad, for an async API).  Maybe some request fails with a specific error ? (grpc::StatusCode::UNAVAILABLE ?)

I suppose there is a way to notify the client that he should not enqueue new requests (or that he can now start pushing new requests).   `ChannelInterface::NotifyOnStateChange` looks a good candidate, except that I could not find a value from `grpc_connectivity_state` that would represent "stop pushing new requests, bro, I'm busy".  Could it be classified as GRPC_CHANNEL_TRANSIENT_FAILURE ?

Thanks for your help

[Vijay Pai]

There is a queue, but it is not hidden. Notice that the last argument in the actual start of the RPC is a CompletionQueue. Starting an operation records some state about it in the CQ, in addition to actually creating a set of operations that are activated in the gRPC C++ library. Operations will take longer to actually complete if they're under pressure and that will be seen when your CompletionQueue::Next calls (not shown in your code snippet, but which is in the AsyncCompleteRPC function executed on a different thread) dribble out one tag at a time. That's the API function that blocks the application based on actual progress.

The channel connectivity state functions refer entirely to the connectivity state diagram at https://github.com/grpc/grpc/blob/master/doc/connectivity-semantics-and-api.md . Failure states there mean true connectivity failures ("Sorry, sister, there's been a real problem."), not just backpressure.

Regards,

Vijay

--
You received this message because you are subscribed to the Google Groups "grpc.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email to grpc-io+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/09f26845-a393-4a30-9d6a-2bf8d9e07e03%40googlegroups.com.

[Frédéric De Jaeger]

Thanks for your reply.

My point is really about the enqueuing of new requests. I imagine that callbacks from the CompletionQueue will come slower, if the server is laggy.  I'm interested in what happens before.

So what exactly happens if the client loops like a crazy and enqueue millions of requests ?

There are 3 possible outcomes:

 1) at some point, the enqueuing fails because an internal buffer saturates.

 2) the process dies with an out of memory (well, same as the previous case, in a sense, but more difficult to recover).

 3) PrepareAsyncSayHello (or StartCall) blocks the caller until some resource becomes available.

Choose your poison.  What happens ?

What is the design pattern to implement backpressure on the client with the c++ interface ?  Shall I control manually how much flying request I have ?

In golang, for instance, I presume the client goroutine is suspended if the outgoing tcp stream is full (so this implements the 3rd outcome and is the perfect semantic)

To unsubscribe from this group and stop receiving emails from it, send an email to grp...@googlegroups.com.