more than 65k connections to upstream cluster

Katka Švecová

unread,

Jul 13, 2022, 7:01:22 AM7/13/22

to envoy-users

Hi,

while using envoy as a TCP proxy, I would like to allow more than 65k connections via one listener to the backend cluster. The cluster consists of several servers so in theory this should not be a problem (TCP in theory allows 65k connections on each of the dst IP/port from the same src IP), however it seems in reality it is not that easy (https://blog.heroku.com/sockets-in-a-bind).

Is there any way how to achieve this with envoy? As far as i know, there is not a possibilty to bind a cluster to several source addresses, only one is allowed. Would enabling SO_REUSEADDR in "socket_options" of the cluster bind config help with this problem? Or are there any other options I missed?

Thank You,

Katka

Katka Švecová

unread,

Jul 13, 2022, 7:34:37 AM7/13/22

to envoy-users

OK, so it seems that the problem is with 'bind before connect", which can be easily avoided by not specifying bind address in the cluster configurationa and just letting envoy to choose one automatically. Then the bind() is not called and I should be able to make 65k connections to every upstream server, is that correct?

Thank you,

Katka

Matt Klein

unread,

Jul 13, 2022, 11:23:41 AM7/13/22

to Katka Švecová, envoy-users

IIRC Envoy does not bind by default for TCP upstream connections, so I think it should just work? Are you seeing otherwise?

--
You received this message because you are subscribed to the Google Groups "envoy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to envoy-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/envoy-users/edaa8511-4fd6-403e-88dd-d13e74d41815n%40googlegroups.com.

Katka Švecová

unread,

Jul 15, 2022, 5:48:33 AM7/15/22

to envoy-users

Hi Matt,

after all it seems to be working fine and envoy is successfully reusing src port without problems.