TCPProxy + host routing..?

[Stephen Murphy]

Hello Envoy users,

I am using TCPProxy to enable TLS between 2 pods. The destination pod is a statefulset using a headless service, so that I need to route traffic to each specific statefulset's DNS (worker-0.worker-headless-svc, worker-1.worker-headless-svc, etc).

Using hostAliases on the originating pod, I can route these multiple host destinations to a single envoy listener. Is there a way to add filtering to the envoy listener that can detect the host used by the originating app and then use this host to route to a specific cluster/statefulset?

 

The traffic from the originating app reaches the TCPProxy in http, so I cannot use TlsInspector. I don't see solution after reading FilterMatchChain documentation.

Ideally, instead of TCPProxy, I would use HttpConnectionManager to accomplish this routing, but HttpConnectionManager results in a 400 Bad Request (see conversation "HPE_INVALID_METHOD when using http_connection_manager").

[Stephan Zuercher]

I went back and looked at the TCP dump for your attempt using the HTTP Connection Manager and it doesn't look like the request is an HTTP request. That explains the 400 error, specifically because the first bytes of data received by Envoy aren't a valid HTTP method (GET, POST, etc). If the request is meant to be HTTP, I'd start with debugging what's going wrong there, since it's much easier to route HTTP requests than TCP connections. 

If this isn't actually HTTP, then another option, which I haven't tried, is to combine the TLS inspector, the SNI cluster filter and the TCP proxy filter. Provided your clients use the SNI extension in TLS, the SNI cluster filter looks like it should provide the server name from SNI to the TCP proxy. The TCP proxy will look for a cluster whose name is the server name and forward the connection there. You'd have to configure a cluster per stateful set instance.

Similarly, you could use the TLS inspector and a filter chain match on SNI to select a TCP proxy configuration that forwards to a specific cluster. You'd have to have a filter chain per stateful set instance and a cluster per stateful set instance to get this to work.

 I'm not sure either of those is quite what you want since it requires the client to choose which stateful set instance to route to by setting the SNI name.

Stephan

--
You received this message because you are subscribed to the Google Groups "envoy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to envoy-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/envoy-users/f2e074ca-2078-4ccf-96a5-2b7c78827934n%40googlegroups.com.