Storing server side data during a request's lifetime

[Amit Saha]

Hi all,

I am playing around with the interceptor implementation  (https://github.com/grpc/grpc/issues/8767) and wanted to start exporting metrics such as per-request latency. To this end, is the servicer context the right place to keep track of such data? For example:

class MetricInterceptor(UnaryServerInterceptor, StreamServerInterceptor):

     def intercept_unary(self, request, servicer_context, server_info, handler):

         response = None

         try:

             servicer_context.start_time = time.time()

             print('I was called')

             response = handler(request)

         except:

             e = sys.exc_info()[0]

             print(str(e))

             raise

         print('Request took {0} seconds'.format(time.time()-servicer_context.start_time))

         return response

...

Thanks for any help.

Best Wishes,

Amit.

[Nathaniel Manista]

On Fri, Sep 29, 2017 at 6:35 AM, Amit Saha <amits...@gmail.com> wrote:

Hi all,

I am playing around with the interceptor implementation  (https://github.com/grpc/grpc/issues/8767) and wanted to start exporting metrics such as per-request latency. To this end, is the servicer context the right place to keep track of such data?

I'm tempted to say no? I think we want to avoid supporting use of grpc.ServicerContext objects as places to store arbitrary data.

For example:

class MetricInterceptor(UnaryServerInterceptor, StreamServerInterceptor):

     def intercept_unary(self, request, servicer_context, server_info, handler):

         response = None

         try:

             servicer_context.start_time = time.time()

             print('I was called')

             response = handler(request)

         except:

             e = sys.exc_info()[0]

             print(str(e))

             raise

         print('Request took {0} seconds'.format(time.time()-servicer_context.start_time))

         return response

Why not just use a local field? Is there some subtlety I'm missing in your example?

-N

[Amit Saha]

I decided to use server context because i assumed that it would be a request local object. When my server is handling concurrent requests, i would not want to use a local variable to keep state (without any locking i.e). 

Please correct me where I am going wrong.

-N

[Amit Saha]

I found that this is not the case, a local variable defined here isn't going to be shared (A class attribute of MetricInterceptor would be). Is there a high level document somewhere explaining more about how the server hands off the incoming request to the relevant servicer? Is https://github.com/grpc/grpc/blob/master/src/python/grpcio/grpc/_server.py#L672 the right place to start looking to understand more?

 

-N

[Nathaniel Manista]

On Fri, Sep 29, 2017 at 5:14 PM, Amit Saha <amits...@gmail.com> wrote:

I decided to use server context because i assumed that it would be a request local object.

Instances of grpc.ServicerContext are indeed per-RPC objects, but their role is to afford certain data and behaviors to service-side applications and to accept certain other data from service-side applications. We'd be reluctant to expand their scope to include "a place to put stuff until later".

When my server is handling concurrent requests, i would not want to use a local variable to keep state (without any locking i.e).

If your application has state that you only need to last as long as the evaluation of a single behavior (per-RPC state), why is a behavior-local field ("local variable") not the right place to store that state?

-N

[Nathaniel Manista]

On Sat, Sep 30, 2017 at 5:56 PM, Amit Saha <amits...@gmail.com> wrote:

Is there a high level document somewhere explaining more about how the server hands off the incoming request to the relevant servicer? Is https://github.com/grpc/grpc/blob/master/src/python/grpcio/grpc/_server.py#L672 the right place to start looking to understand more?

I recommend looking at both the documentation of the grpc package (particularly grpc.GenericRpcHandler and grpc.RpcMethodHandler) as well as at the code generated in _pb2_grpc.py files (to understand how that code translates your application's calls into calls on the grpc package).

-N

[Amit Saha]

I was confused (didn't think it through) about the local variables somehow being "shared" across calls.  I have been able to verify and got my thinking straight about that not being the case. 

 

-N

[Amit Saha]

Thanks for the suggestions.

 

-N