UDP Proxy Filter ephemeral ports management

[Eran Rom]

I would like to suggest a new feature for the UDP proxy filter to help overcome the ephemeral ports exhaustion problem.

Background:

According to [1], a way to overcome the known ephemeral ports exhaustion problem is to use the REUSEADDR option together with bind and connect. This allows to extend the limited 2-tuple port range to 4-tuple.

This, however, can lead to multiple sockets with the same 4-tuple to be created, resulting with a newer socket "overshadowing" previous ones.

Proposed Solution:

While [1] proposes several solutions, I would like to propose a different one which seems simpler. The idea is to let envoy manage a set of configurable ephemeral ports ranges as follows:

1. The UDP proxy filter configuration is added with an ephemeral port ranges entry (e.g. 1024-5,000, 10,000-50,000)

2. The filter manages a <host addr, port queue> map.

3. As soon as the socket is created in UdpActiveSession::writeUpstream we use the host addr to look up the associated port queue from the map and pop a "free" port. The port is placed back in the queue when the session ends.

4. Getting a port we set the SO_REUSEADDR socket option and bind to the port.

5. In case we are out of ports we fail the request increasing e.g. upstream_cx_overflow

Some more details:

1. Having a <host addr, port queue> map to manage the ports assumes that the same host addr does not appear in more than one cluster. 

2. In order to have a lockless data structure the idea is to split the range between the different filter workers. For simplicity, say that our range is 10,000-50,000 and we have 8 workers, the worker with index 1 will use the range 10,000-14,999, the worker with index 2 15,000-19,999 and so on...

A PoC implementation seems to do the job.

Comments are welcome.

[1] https://blog.cloudflare.com/how-to-stop-running-out-of-ephemeral-ports-and-start-to-love-long-lived-connections/#outgoing-connections-on-linux-part-2-udp

[Yan Avlasov]

Hi Eran,

Please file this request as an Issue on https://github.com/envoyproxy/envoy You can also submit a PR if you already have PoC

--
You received this message because you are subscribed to the Google Groups "envoy-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to envoy-dev+...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/envoy-dev/cf8cac17-777d-4e88-9493-e86b461d2834n%40googlegroups.com.