Injecting Events in Grpc Completion Queue

[Satyam Shekhar]

Hello,

I am trying to build a high-performance server with GRPC. As recommended in performance notes for cpp, I wish to create 1 completion-queue per CPU and tie 1 completion-queue per thread. I'd prefer to avoid any other thread in my system. All threads would continuously poll from their respective completion-queue and process read events.

The server also has interactions with other IO systems - say Kafka. For this model to work efficiently, the server should be able to schedule other events (say Kafka write finished) on completion-queue to avoid polling multiple IO services. 

Grpcpp CompletionQueue interface does not support scheduling events. Core surface API has that interface but that is not publicly visible via Bazel. 

I am looking for guidance on the following questions -

1. Is this approach recommended? 

2. If yes, how can I schedule events on CompletionQueue?

3. What other alternatives should we consider? 

Regards,

Satyam

[Mark D. Roth]

We don't intend to support applications injecting their own events into a gRPC completion queue.  In fact, the only reason that mechanism exists under the hood is that it's part of our effort to transition away from the completion queue-based API.  Read on for details.

We don't currently have a mechanism that allows integrating gRPC into an external event loop, although we are in the process of making some changes that will allow that.  Basically, we are currently working on introducing an EventManager API and changing C-core to use that API for polling.  We will provide an EventManager implementation based on libuv, but applications that wish to integrate gRPC with an external event loop will be able to supply their own EventManager implementations that wrap their external event loop, in which case gRPC can be driven by that external event loop.

Once we move to EventManager-based polling, we will be able to make the new reactor-based C++ async API available in OSS (we're already using it internally), and we will encourage people to use that API instead of the current completion queue-based API.  The reactor API is much easier to use for most applications.

The current ETA for making the EventManager API available and finishing the libuv-based EventManager implementation is the end of April, so stay tuned for more information.

I hope this information is helpful!

--
You received this message because you are subscribed to the Google Groups "grpc.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email to grpc-io+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/CAAy_rtHxP7YmEMJD5wTDKG%3DDsXW41qWZzOzHczcPgoZDo4H0bg%40mail.gmail.com.

--

Mark D. Roth <ro...@google.com>
Software Engineer
Google, Inc.

[Satyam Shekhar]

Thanks for the update. This is really helpful.

Can you provide some samples of the EventManager API and reactor API? That will enable us to design our system to be amenable to future changes.

Regards,

Satyam

[Mark D. Roth]

We haven't yet published gRFCs for either the EventManager API or for the C++ reactor-based API, but look for those to be available within the next couple of months.

For the C++ reactor-based API, we do have some tests you can look at as examples.  For server-side, see here, and for client-side, see here.

I hope this info is helpful!

[priy...@gmail.com]

Consider a async GRPC based key value store that accepts events on a CompletionQueue.To service those events, it must do non-GRPC IO (perhaps read data from the local file system). What is the recommended pattern for integrating such non-GRPC async IO into an async GRPC server?

The ideas we are considering are:

- inject events onto the GRPC completion queue: seems that you are recommending against that.

- post from the CompletionQueue event loop onto an external event loop: in our tests, posting to an external event manager seems to really kill our observed through (admittedly early numbers though).

Any alternatives we are missing?

To unsubscribe from this group and stop receiving emails from it, send an email to grp...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/CAAy_rtHxP7YmEMJD5wTDKG%3DDsXW41qWZzOzHczcPgoZDo4H0bg%40mail.gmail.com.

--

Mark D. Roth <ro...@google.com>
Software Engineer
Google, Inc.

[Mark D. Roth]

The only other alternative you could use today would be to just have separate sets of threads handling the gRPC events and the external events, so that you don't really try to combine the two event loops at all.  But that may or may not work well, depending on how your application is structured.

We believe that the new EventManager API we're currently working on will provide a better solution to this problem.  But if you need something before that's ready, then you may need to go with either the above or with the "post from the CompletionQueue event loop onto an external event loop" approach you suggested.

To unsubscribe from this group and stop receiving emails from it, send an email to grpc-io+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/fa127ce4-c1e3-4a87-9b86-01f1bd6b9a57%40googlegroups.com.