StatusCode.CANCELLED with asynchronous RPC in Python

[Gerold Ruediger]

Hello grpc users,

I am new to grpc, and I cannot get asyncronous RPC calls to work as expected.

The example below defines a RPC call with name "delayed", which should return the value 42 after "time.sleep(1)".

I get this "StatusCode.CANCELLED" error when invoking the client (after having started the server):

$ python client.py 

Exception in thread Thread-2:

Traceback (most recent call last):

  File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner

    self.run()

  File "/usr/lib/python2.7/threading.py", line 754, in run

    self.__target(*self.__args, **self.__kwargs)

  File "/usr/local/lib/python2.7/dist-packages/grpc/_channel.py", line 716, in channel_spin

    completed_call = event.tag(event)

  File "/usr/local/lib/python2.7/dist-packages/grpc/_channel.py", line 172, in handle_event

    callback()

  File "/usr/local/lib/python2.7/dist-packages/grpc/_channel.py", line 313, in <lambda>

    self._state.callbacks.append(lambda: fn(self))

  File "client.py", line 9, in cb

    print("Test client received: " + str(future.result().value))

  File "/usr/local/lib/python2.7/dist-packages/grpc/_channel.py", line 279, in result

    raise self

_Rendezvous: <_Rendezvous of RPC that terminated with (StatusCode.CANCELLED, Cancelled)>

If I reduce the delay from 1 second to 0.05 seconds, most of the RPC attempts succeed, and only a few fail. What am I missing to make this example work, such that the future notifies its callback method even if the server needs more than a second to reply?

Thank you

Gerold

These are the files required to reproduce the problem:

+++ test.proto:

syntax="proto3";

service Test {

   rpc delayed(Empty) returns (Result) {};

}

message Empty{

}

message Result{

  int32 value = 1;

}

+++ server.py:

from concurrent import futures

import time

import grpc

import test_pb2

import test_pb2_grpc

class Test(test_pb2_grpc.TestServicer):

  def delayed(self, request, context):

    time.sleep(0.5)

    return test_pb2.Result(value=42)

def serve():

  server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))

  test_pb2_grpc.add_TestServicer_to_server(Test(), server)

  server.add_insecure_port('[::]:50051')

  server.start()

  try:

    while True:

      time.sleep(10)

  except KeyboardInterrupt:

    server.stop(0)

if __name__ == '__main__':

  serve()

+++ client.py:

from __future__ import print_function

import grpc

import test_pb2

import test_pb2_grpc

def cb(future):

  print("Test client received: " + str(future.result().value))

def run():

  channel = grpc.insecure_channel('localhost:50051')

  stub = test_pb2_grpc.TestStub(channel)

  fut = stub.delayed.future(test_pb2.Empty())

  fut.add_done_callback(cb)

if __name__ == '__main__':

  run()

[Nathaniel Manista]

Thank you for including your code with your question.

I'm nearly certain that the problem is right here: your RPC is still in progress when your run() function runs out of statements to execute, and when a function runs out of statements to execute, the objects that are only referenced from the functions local scope get garbage collected*, and when a grpc.Future object gets garbage collected, any ongoing RPC associated with the object gets cancelled, and when a grpc.Channel object gets garbage collected, all ongoing RPCs associated with the object get cancelled. So the right solution is probably "do something to avoid returning from your run function while the RPC is ongoing". If this were real code I'd suggest just making a blocking RPC call, but since I suspect you're trying to use this code to teach yourself about making asynchronous calls, try out things like time.sleep, <your future object>.result, <your future object>.exception, <your callback object>.block_until_callback_called, and so forth.

if __name__ == '__main__':

  run()

-Nathaniel

*One should never code depending upon garbage collectors being deterministic, but when debugging it's okay to talk about them typically or probably deleting objects at certain times.

[Gerold Ruediger]

Thank you, your suggestion solves the problem.

Gerold

Virenfrei. www.avast.com