Multiple servers on different ports on the same CompletionQueue

[Austin Schuh]

I'm trying to share a CompletionQueue between multiple servers on different ports. The desired setup is as follows (and I'm very open to alternative designs if I'm doing something conceptually wrong.)

public client (x509) -> firewall -> server : 5001

                                                    server : 5002 (unsecure, or basic SSL) <- internal client

The basic idea is that I have APIs that I want to expose to external clients, and APIs that I only want to expose to other services inside our datacenter.  I don't want either type of client to be able to access the APIs for the other type of client.

I read https://groups.google.com/forum/#!topic/grpc-io/iag1a5RPez0 which asserts that it's possible to create multiple servers on different ports.  All I can see in ServerBuilder is various ways to allow multiple servers to *all* listen on multiple ports.  I want to have server 1 listen on port 1, and server 2 listen on port 2.  I've got an async application right now and want to share one CompletionQueue between the two services.  Is this possible?

I'm starting my server using:

 void *MakeTag(::std::function<void(bool)> function) {

   return static_cast<void *>(new ::std::function<void(bool)>(function));

 }

 void RunTag(void *tag, bool ok) {

   (*static_cast<::std::function<void(bool)> *>(tag))(ok);

 }

 void FreeTag(void *tag) {

   delete static_cast<::std::function<void(bool)> *>(tag);

 }

...

   ::grpc::ServerBuilder builder;

   builder.AddListeningPort(server_address, MakeServerCredentials());

   builder.RegisterService(&service);

 

   ::std::unique_ptr<::grpc::ServerCompletionQueue> cq =

       builder.AddCompletionQueue();

 

   ::std::unique_ptr<::grpc::Server> server(builder.BuildAndStart());

   new Request(server.get(), &service, cq.get());

   void *tag;

   bool ok;

   while (true) {

     GPR_ASSERT(cq->Next(&tag, &ok));

 

     grpc_core::ExecCtx exec_ctx;

     // Null tag means no action.

     if (tag) {

       RunTag(tag, ok);

       FreeTag(tag);

     }

   }

My workaround today is to create 2 ServerBuilders in 2 threads.  That feels wrong.

Thanks,

  Austin

[Vijay Pai]

This is an interesting phenomenon. Indeed, there is currently no C++ API to allow the same CQ to be added to 2 different servers as the ServerCompletionQueue, so if you really need 2 servers listening on two different ports then you're going to have 2 separate ServerCompletionQueue's and most likely need to Next them using separate threads. This issue will be mitigated when we have the C++ callback API ready for use as there is no explicit CQ in that case.

Regards,

- vjpai