Envoy as a node-scoped agent

[Bruno Palermo]

Hi there

I watched Tony Allen's envoycon talk about envoy as a node-scoped agent and was playing with the concept.

The question I have is related to how to propagate the pod identity.

From the traffic flow slide in the presentation I assume nftable + OriginalDst filter is being used. And because we are binding both the listener and the upstream cluster within the pod's network namespace, we have an upstream cluster for each pod where we can define the UpstreamTlsContext and forward each pod's identity.

 Assuming my understanding is correct, and I know the presentation was focused on L4, I wonder how we could extrapolate the use case for L7 as well. From what I can see I would have to duplicate upstream clusters so I could have the appropriate UpstreamTlsContext associated depending on the source pod.

Any insights would be appreciated.

Thanks!

---

Confidentiality note: This e-mail may contain confidential information from Nu Holdings Ltd and/or its affiliates. If you have received it by mistake, please let us know by e-mail reply and delete it from your system; you may not copy this message or disclose its contents to anyone; for details about what personal information we collect and why, please refer to our privacy policy.

[Tony Allen]

Hi Bruno,

You might be able to make use of transport socket matching. I think you’ll want some combination of:

Header to metadata filter:

https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/header_to_metadata_filter

Transport socket matching based on metadata:

https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/cluster/v3/cluster.proto#envoy-v3-api-msg-config-cluster-v3-cluster-transportsocketmatch

You can use a single cluster and this will allow envoy to choose a transport socket configuration based on filter state metadata. You would just need to ensure the metadata is different for each source pods UpstreamTLSContext in the matcher, which you can do a number of ways. The header-to-metadata filter is just one way. 

LMK if you have any questions about that approach. 

-Tony

[Bruno Palermo]

Thanks Tony,

I wasn't aware of this feature, I totally get what you are suggesting.

I will give it a try and let you know if I run into any issues.

Thanks!