controlling GRPC client prefetch

[Asheley Shawn Lee]

I'm looking for a way to control how many messages a client prefetches from a server.

For example: If a client is processing through a message every 60 seconds and the server has 100,000 messages to send to the client then the client will prefetch all 100,000 of the messages as it continues to process the messages asynchronously.

Is there a way to reduce the size of the prefetch buffer on the client?

I'm trying to use grpc.http2.lookahead_bytes option with no impact.

In my testing I'm getting about 4MB of client cached messages before the client doesn't

ask for any more.

I have a POC that I can transfer over from my work machine to this forum but it's basically just a server with one method that yields as fast as possible and a client that receives the message and just sleeps for 60 seconds.

[Asheley Shawn Lee]

This looks very close to my issue.

https://stackoverflow.com/questions/64050027/how-to-configure-grpc-http-2-flow-control-in-python

[Asheley Shawn Lee]

Looks like I was able to find a solution with grpc.insecure_channel options.

I have to use two options:

("grpc.http2.lookahead_bytes", 8 * 1024),

("grpc.http2.bdp_probe", 0),

bdp stands for "Bandwidth Delay Product" and appears to be used by the flow control algorithms and needs to be disabled.

[dfa...@google.com]

Can you explain your reasoning for wanting to override these settings?  BDP and this type of "prefetching" is used to improve performance.  Setting these options may not be the best way to solve the problems you are facing, as they will impact all traffic on those connections.  It may be preferable, e.g., to have the client indicate explicitly to the server how many messages should be outstanding at any point in time, and then ack the messages as it is done processing them.