Running multiple instances of the same service within a grpc server

[fede...@gmail.com]

Hello everyone.

I was trying to see if it's possible to run different instances of the same service on a single GRPC server, but it looks like I'm not able to do so. So I was wondering if I was doing anything wrong with my test, or if it's not possible at all.

My test is based on examples/python/multiplex from grpc repo:

Service:

class _GreeterServicer(helloworld_pb2_grpc.GreeterServicer):

    def __init__(self, greeter):
        self.greeter = greeter

    def SayHello(self, request, context):
        return helloworld_pb2.HelloReply(
            message='Hello, {}! This is {}!'.format(request.name, self.greeter))

Server:

def serve():
    server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
    helloworld_pb2_grpc.add_GreeterServicer_to_server(_GreeterServicer("John"),
                                                      server)
    helloworld_pb2_grpc.add_GreeterServicer_to_server(_GreeterServicer("Jim"),
                                                      server)
    server.add_insecure_port('[::]:50051')
    server.start()
    server.wait_for_termination()

Client:

def run():
    for _ in range(10):
        with grpc.insecure_channel('localhost:50051') as channel:
            greeter_stub = helloworld_pb2_grpc.GreeterStub(channel)
            greeter_response = greeter_stub.SayHello(
                helloworld_pb2.HelloRequest(name='you'))
            print("Greeter client received: " + greeter_response.message)

Since I'm opening a new channel for each iteration, I was expecting to get an output with a mix of "Hello, you! This is Jim!" and "Hello, you! This is John!", but instead I'm getting only:

Greeter client received: Hello, you! This is John!
Greeter client received: Hello, you! This is John!
Greeter client received: Hello, you! This is John!
Greeter client received: Hello, you! This is John!
Greeter client received: Hello, you! This is John!
Greeter client received: Hello, you! This is John!
Greeter client received: Hello, you! This is John!
Greeter client received: Hello, you! This is John!
Greeter client received: Hello, you! This is John!
Greeter client received: Hello, you! This is John!

that is, the first service of GreeterServicer I added to the server, which supposedly ignore the second servicer instance.

So, my question is if it's even possible to do something like this on a single server and if there is a best practice to handle such scenarios where I would want two mostly identical services, parameterized differently: different servers load balanced (thus, following https://grpc.io/blog/grpc-load-balancing best practices)? Multiple threads implemented by me within the service?

In my particular scenario, the parameters I'd like to pass are some credentials to be used within the service implementation, and my goal would be, then, to have these multiple identical services running concurrently, with different credentials, so that the end user has no idea there are multiple instances, when making requests.

Thank you very much,

Federico

[Mark D. Roth]

When you make an RPC on the client, it sends the RPC service and method name to the server as the HTTP/2 path.  When you register the service implementation on the server, it is essentially stored in a map keyed by that path.  When the server receives a request for a particular path, it looks up the path in that map to determine which service implementation to use.  Given that, hopefully it is clear why it is not possible to register two service implementations for the same service and method on the same server.

Yes, you could have two different servers listening on two different machines that have different implementations for the same service and then have your clients load-balance between them.  However, I suspect that wouldn't actually help you, because you would not have any explicit control over which requests go to which server.  If you actually don't care which requests go to which server, then you could probably more easily do the same thing by having a single server implementation that simply checks which implementation it used on the last request and uses the opposite one on the current request.

When you say that you want to pass credentials as parameters, would these be parameters in the RPC request passed from the client?  If so, I think the right way to handle that is to have a single server implementation that looks at the parameter and invokes the right underlying implementation based on which parameters were passed.

I hope this information is helpful.

[fede...@gmail.com]

Thank you very much for the explanation, very helpful.

I actually don't care which actual server is going to serve the client requests, because in my mind the clients should be agnostic.

Those credentials parameters I talked about should be passed as configuration to the server, used by client objects instantiated within the service and these objects should remain unique for each credential and I hoped that it could work by instantiating multiple servicers.

I guess that something similar to your proposed approach could be a pretty easy solution.