Envoy dedup endpoints and cause traffic imbalaned

[chzh....@gmail.com]

Hi Envoy experts,

We have seen that Envoy seems dedup the endpoint if the address IP + port combo is the same. We run into this scenario as our upstream endpoints could share same <node IP + port> from k8s cluster. Though each upstream endpoint has unique <pod IP + port> combo, the address exposes to service discovery is node IP, as pod IP is private. There is a IPTable on the k8s node to randomly load balance traffic if pods from same upstream cluster are colocated. As you can see, with Envoy dedup behavior, the traffic will become imbalanced eventually.

For example, the upstream cluster is like

Endpoint A -> 172.168.0.1:1234

Endpoint B -> 172.168.0.1:1234

Endpoint C -> 172.168.0.2:1234

We expect each of the endpoint gets 1/3 of traffic. However, with deduping, Endpoint A and B will get 25% of traffic but Endpoint C gets 50%.

Our current solution is to ask the control plane (a shim) to dedup the endpoints first and then applying weight accordingly to each endpoint. So we send 2X of traffic to 172.168.0.1:1234 but 1X to 172.168.0.2:1234. As a result, each endpoint will get 1/3 of traffic, which is balanced.

However, from here, I read that weighted endpoint is exclusively working with zone aware routing. We also use zone aware routing so I am seeking for a solution to work for both cases.

Any suggestion will be appreciated. Thanks.

Regards,

Austin