Dynamic Metadata is not passed to HTTP filters

[Michael Wiederhold]

I’m currently using the proxy protocol v2 filter that was recently added in v1.15. Our use case requires parsing the TLV section which is supported by Envoy and puts the parsed data into the dynamic metadata section. Our goal was to read from the dynamic metadata at the HTTP level in order to propagate what is in the proxy protocol TLV section to downstream services. However, when we tried to do this we noticed that nothing was sent. Upon digging into the source code we found that there are three levels of filters; listener, network, and http. The dynamic metadata is propagated from the listener filters to the network filters, but not from the network filters to the http filters. I have two questions:

1. Is this intended and if so what is the reason for not propagating metadata through all filter levels?
2. If there is no reason for not propagating the dynamic metadata would the community be ok with me making this change?

[Lizan Zhou]

Yes this is intended. There are two level objects underlying, L4 connection and L7 stream, listener filter and network filter are tied to L4 connections, and HTTP filters are tied to L7 streams. They aren't 1:1, same connection can have multiple streams in keep-alived HTTP/1.1, HTTP/2 or any other protocols. If you're writing a filter, you should be able to access proxy protocol filter extracted metadata through connection() method, is that what you're looking for?

--
You received this message because you are subscribed to the Google Groups "envoy-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to envoy-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/envoy-dev/914323a9-4b0a-4693-9fe8-1cdd30968ec2n%40googlegroups.com.

[Michael Wiederhold]

Lizan,

Thanks for getting back to me. I'm currently working on an integration with AWS Private Link. They add an endpoint id in the TLV section of the proxy protocol v2 blob and I need to propagate that downstream preferably by appending an http header to http requests on that connection. The listener filter for proxy protocol v2 puts that data into the dynamic metadata section and it sounds like what you're saying is that in order to access this I'd need to add a new filter which accesses the connection() method? Is that correct or is there any other way you might recommend doing this?

- Mike

[Lizan Zhou]

Then I think what you want to do is similar to https://github.com/envoyproxy/envoy/pull/11858, instead of the HTTP dynamic metadata you want connection metadata. Can you open an issue in Envoy so we can track there? The change will be very similar to this one.

[Michael Wiederhold]

Thanks Lizan. I'll open an issue and will also work on getting a PR put up.

- Mike

[Seikun Kambashi]

Hi Lizan,

Thanks for your previous responses. I just wanted to clarify, when you mentioned that we can do something similar to https://github.com/envoyproxy/envoy/pull/11858, are you suggesting we modify the same header_formatter with a new field_extractor_ for connection dynamic metadata? From what I understand it looks like this header formatter is only given the HTTP StreamInfo. So would that mean we would have to find a way to thread the connection() information from higher up, is that right?

Or do we still need to create a new L7 HTTP filter if we want this functionality? I'm trying to understand where this filter would then fit into our Envoy config. Would it make sense / is it possible to create an L7 filter to grab the proxy-protocol filter's extracted metadata and attach it as an HTTP header to the request, and then stick that filter into the HTTP connection manager network filter's http_filters?

Envoy issue for tracking: https://github.com/envoyproxy/envoy/issues/12593

Thanks,

Seikun Kambashi

[Seikun Kambashi]

Alternatively, is it possible to expose the connection metadata to the lua filter?