Accessing Current Connections to Server

[Nicholas Bunn]

Hi everyone,

I'm looking to implement a rate limit interceptor for my gRPC services, allowing for custom rate limiting based on the service's latency. I don't need any advanced rate limit rules, just evaluating how many connections have been made in total (no IP-based checks). I need to do this in both Python and Go for my application. I was wondering if there is a way to access the connection information of the server in order to see how many connections the server is currently responding to? Alternatively, does anyone have any suggestions for an alternate approach to this?

Thanks in advance,

Nic

[Sergii Tkachenko]

Hi Nic,

> I was wondering if there is a way to access the connection information of the server in order to see how many connections the server is currently responding to

One way that comes to mind is adding channelz service to the server, and querying it for the data. Related link: https://cloud.google.com/traffic-director/docs/observability-proxyless#admin-interface (`AdminInterface` includes channelz).

Otherwise, Java provides `ServerBuilder.addTransportFilter()`, which can be used to track connections. Look up if there's something similar in Python/Go.

> Alternatively, does anyone have any suggestions for an alternate approach to this?

If you're using gRPC xDS, I'm working on adding Rate Limit Service support for server-side global rate limiting. This allows to offload throttling decisions to an external server.

Best regards,

Sergii

[Nicholas Bunn]

Thanks Sergii! 

For some preliminary tests I've come right with just implementing a simple counter in the interceptor. This works fine for the Go implementation but Python has given some inconsistent results depending on the server configuration (call limit vs. number of workers vs. number of concurrent requests). I'll definitely have a look into the channelz service and transport filters to try and get around this. Otherwise, I've got a global rate limit implemented in my Envoy gateway, might look at adding some rules in here at a service level or throwing Istio onto my services. 

Thanks again,

Nic