Java grpc client behaviour on hitting H/2 max concurrent streams

[Gaurav Poothia]

Hello!

I have a situation where the server sets H/2 max concurrent streams intentionally low (e.g. 16 or 32) for good reason (that are somewhat orthogonal to gRPC so I will skip the details)

The behavior I seem to be observing is that Java gRPC client stalls when it hits that limit and waits for streams to free up before issuing more requests.

Is this the expected behavior?

I was hoping for a knob that allows me to instruct gRPC to dial additional H/2 connections when it hits the limit. Turns out the .Net implementation has something exactly  like that
https://docs.microsoft.com/en-us/aspnet/core/grpc/performance?view=aspnetcore-5.0#connection-concurrency

Does Java gRPC stack have an equivalent? 

It would be pretty useful in seamlessly handling service to service communication where server can support a higher gRCP req/sec but only if the client were to dial more H/2 connections.

I say "seamlessly" because of course we can write code to somehow dial more subchannels to server but we want the client to do this automatically without programmer picking hardcoded connection count.

Thanks in advance!

Gaurav

[Yuri Golobokov]

Hi Gaurav,

For Java you might find useful the GAX channel pool:

https://github.com/googleapis/gax-java

https://github.com/googleapis/gax-java/blob/master/gax-grpc/src/main/java/com/google/api/gax/grpc/ChannelPool.java

--
Yuri

[Eric Anderson]

Yes, this is on purpose, and is actually expected per the HTTP/2 spec. We recognize there are situations where that is not the correct behavior and more connections should be opened, although that point may be before hitting MAX_CONCURRENT_STREAMS. Determining when to open more connections is a bit involved as it depends on your load balancer approach. It isn't helped by the fact that MAX_CONCURRENT_STREAMS can be dynamic and servers are allowed to set MAX_CONCURRENT_STREAMS to 0 (momentarily); that's actually something C core can do when under extreme memory pressure. Opening more connections can make overload situations worse. Anyway, it is complicated. Clients that just open more connections tend to assume high-scale internet load balancers that can accept ~infinite number of connections and use hard-coded MAX_CONCURREN_STREAMS.

Using a solution like ChannelPool (linked in Yuri's email) is the suggested workaround in Java; it is a bigger issue in some other languages where you don't have a small Channel interface that the stubs use.

--
You received this message because you are subscribed to the Google Groups "grpc.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email to grpc-io+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/a687afdc-6c52-449f-85ab-5c9d83543f08n%40googlegroups.com.