Aggregation/Parallel Asynchronous Calling

[Nicholas Bunn]

Hi there, I'm developing an application which will aggregate information from various services and return it to the client. The system will be distributed and I'd like to implement asynchronous communication where possible to avoid lengthy processes stalling the system. 

I think I understand the "asynchronous" implementation of gRPC in terms of streaming, where it's asynchronous considering that you don't necessarily wait for the invoked service to complete its process before getting a response. I'm looking for the non-blocking aspects of asynchronous communication, so considering the explanation below - I hope someone can provide some insight for me.

Say there exists some aggregating service and four other services from which it will be collecting information (services A, B, C, and D). With standard synchronous calls, the aggregating service would have to query service A, wait for a response, then query service B, wait for a response, and so on. What I am looking for is a way to query all four services in parallel and wait for a callback or notification from each one once they have completed their process, and their information is available. Would one be able to achieve this with streaming? Or is the client still limited to communication with one server at a time (essentially implementing a blocking call)?

Any assistance/insight would be greatly appreciated!

Thanks,

Nic

[Michael Lumish]

What you are describing is a normal thing to do with gRPC, using either unary or streaming methods. Each of those operations is independent and can be started independently, and then when they finish you can aggregate the results. The exact way to do this depends on what features the specific programming language has for handling asynchronous operations. What language do you expect to do this in?

--
You received this message because you are subscribed to the Google Groups "grpc.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email to grpc-io+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/bc93f296-f129-4f23-a4a7-d76e820d28a5n%40googlegroups.com.

[Nicholas Bunn]

For the aggregation services, I was thinking of going with Java (only because I'm most familiar using Java for bigger projects) - but I'm still very open to language changes at this point. I read somewhere that this can be achieved using C# (Or C++, I don't remember which). Do you have any suggestions?

[Michael Lumish]

I primarily work on the Node.js libraries. Maybe someone on the Java team like +Eric Anderson can comment on how this sort of thing can be done in Java.

To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/6a05d9b5-555e-4f91-be94-ebea05dd8c94n%40googlegroups.com.

[Eric Anderson]

On Mon, Sep 21, 2020 at 5:54 AM Nicholas Bunn <nichola...@gmail.com> wrote:

What I am looking for is a way to query all four services in parallel and wait for a callback or notification from each one once they have completed their process, and their information is available. Would one be able to achieve this with streaming? Or is the client still limited to communication with one server at a time (essentially implementing a blocking call)?

Unary vs streaming and blocking vs async (or non-blocking) are separate dimensions. Unary vs streaming is talking about the actual semantic structure while blocking vs async is a detail about the API. So blocking vs async is really a question of "how have gRPC implementations exposed these RPCs."

A large number of gRPC language APIs support don't-consume-a-thread-per-RPC. Go uses blocking, but goroutines avoid your concern. Java has a Future API for unary and an async API for unary and streaming. Kotlin supports coroutines. C# supports async/await, although I'm not as familiar with the streaming API. Node.js is (obviously) async. Python now supports async/await. C++ has an async API, although it is pretty different than what you may be used to (and IIRC the callback API isn't quite ready yet). And there's other implementations as well.

So you are pretty free to take your pick. If you are leading toward Java already, using unary RPCs and Futures is probably easiest. You can use something like Guava's Futures.allAsList() to combine the results and cancel the other RPCs if any one of the RPCs fails. There's many other ways to mix-and-match. Streaming is certainly possible as well, but there's fewer utilities to combine the results so you may have more boilerplate.

[Nicholas Bunn]

Okay great, thanks!

That's helped to clear it up quite a bit for me. Is there a particular reason to avoid sending each RPC call in its own thread?

[Eric Anderson]

On Mon, Sep 28, 2020 at 4:19 AM Nicholas Bunn <nichola...@gmail.com> wrote:

That's helped to clear it up quite a bit for me. Is there a particular reason to avoid sending each RPC call in its own thread?

Not much from gRPC's perspective. Sync vs async APIs can behave a bit differently, depending on how the threading works out, so sometimes one will be a bit faster than the other (although maybe with a latency vs throughput tradeoff). But for most users the difference is probably in the noise.

The main day-to-day reason to not send each RPC on its own thread is mainly the memory cost of the thread itself. At high scale and high performance, other things can come into play, but most application's don't live at that level of scale. If interested, you can look into the "C10k problem" that originated in 1999, servicing 10k connections/clients at once. But you have to realize a lot of information will be out-of-date, as the precise details change year-to-year.

[Nicholas Bunn]

Okay, thanks for the information - I'm still quite new to all of this so I really appreciate your help!