How to configure Envoy to run at both ingress and egress

[jz]

Hello,

Wasn't able to get a full answer in the other user forum so I am try my luck here. Sorry for duplicated messages:

I am looking at how Envoy can be deployed as service to service proxy, and it seems Envoy can run at both at the egress and ingress side. However, I wonder if there is any special configuration needed to run Envoy at that mode, and how the ingress part works internally. For example, if the EDS response returns a list of upstream endpoints (host:port),  will Envoy forward a request to the upstream service endpoints directly, or to the ingress Envoy on the remote host and let it handle from there?

In another words, is it possible to run Envoy in two modes?:

1. caller -> egress Envoy -> ingress Envoy -> service

2. caller -> egress Envoy -> service

And if so, how to setup each mode?

Any pointers are appreciated. 

jz

[Stephan Zuercher]

Envoy has no knowledge of other Envoy instances. If EDS configures a cluster with a set of host:port endpoints, those are the endpoints that will receive requests that are routed to the cluster. Those endpoints can be other Envoy instances but they don't have to be.

--
You received this message because you are subscribed to the Google Groups "envoy-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to envoy-dev+unsubscribe@googlegroups.com.
To post to this group, send email to envo...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/envoy-dev/0fa61d2f-ce6e-4f85-b106-19a76c08326c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[jz]

I see, that makes sense. 

To follow up, if EDS returns a list of upstream Envoy host:port endpoints, how does an ingress Envoy figure out which local port to route to in order to actually reach the cluster endpoint? 

Thanks a lot!

On Thursday, August 23, 2018 at 3:52:51 PM UTC-7, Stephan Zuercher wrote:

Envoy has no knowledge of other Envoy instances. If EDS configures a cluster with a set of host:port endpoints, those are the endpoints that will receive requests that are routed to the cluster. Those endpoints can be other Envoy instances but they don't have to be.

On Wed, Aug 22, 2018 at 2:20 PM, jz <zhanj...@gmail.com> wrote:

Hello,

Wasn't able to get a full answer in the other user forum so I am try my luck here. Sorry for duplicated messages:

I am looking at how Envoy can be deployed as service to service proxy, and it seems Envoy can run at both at the egress and ingress side. However, I wonder if there is any special configuration needed to run Envoy at that mode, and how the ingress part works internally. For example, if the EDS response returns a list of upstream endpoints (host:port),  will Envoy forward a request to the upstream service endpoints directly, or to the ingress Envoy on the remote host and let it handle from there?

In another words, is it possible to run Envoy in two modes?:

1. caller -> egress Envoy -> ingress Envoy -> service

2. caller -> egress Envoy -> service

And if so, how to setup each mode?

Any pointers are appreciated. 

jz

--
You received this message because you are subscribed to the Google Groups "envoy-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to envoy-dev+...@googlegroups.com.

[jz]

Thank you Stephan for the detailed explanation. 

"a single listener that acts as an ingress for the service and just routes inbound requests to 127.0.0.1 on a predefined port", 

Now I realize I may have misunderstood the "sidecar" model, i.e. there is a 1:1 mapping between ingress envoy to the service. I was thinking about a 1:N model, where Envoy can be run as a per-host proxy that multiplexes multiple local service instances on that host. Correct me if I am wrong, look like that's not what Envoy is designed for?

In another words, for service-to-service communication, would it work to deploy one and only one Envoy instance on every host, which multiplexes/demultiplexes all instances in that host?  

On Thu, Aug 23, 2018 at 4:55 PM, Stephan Zuercher wrote:

You have a lot of options.

The service-to-service example presumes that each service instance has an Envoy running as a sidecar with 3 types of listeners (each listener having it's own port):

1. a single listener that acts as an ingress for the service and just routes inbound requests to 127.0.0.1 on a predefined port. All requests to this port are routed to the local service instance.

2. a pair of listeners dedicated to routing http and grpc to other services on the same network. These use the host/authority to route to service clusters (comprised of the host:ports of Envoy ingress listeners on other hosts)

3. a separate egress listener for each external service (e.g. one port for every external service).

The service uses the #2 listeners to communicate with other services you've deployed with Envoy sidecars and uses the #3 listeners to access external services such as databases or third-party APIs (whose clients libraries may not readily allow manipulation of host headers).

[Stephan Zuercher]

This is what I mean by "you have a lot of options." 😀

There's nothing stopping you from running an Envoy per host and having it provide routing for all the services on that host. Another alternative is to run a cluster of Envoys on hosts separate from your service hosts and obtain service endpoint data exclusively via service discovery/EDS.

To unsubscribe from this group and stop receiving emails from it, send an email to envoy-dev+unsubscribe@googlegroups.com.

To post to this group, send email to envo...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/envoy-dev/5fa79cea-9e14-45b7-bfd9-ecce8a768bb9%40googlegroups.com.

[jz]

hmm, I must have missed something. For Envoy to run per host and serve both egress and ingress, kinda going back to my original question, how does the ingress Envoy figure out which local service instance to demultiplex into, since you mentioned the ingress listener will route inbound requests to only a single predefined port (local service instance)? To me, the ingress Envoy must learn about all the host-local instances dynamically, but I don't how that's possible. 

Appreciate you patience on this. 

[Stephan Zuercher]

If you run a single Envoy per host, I would expect there to be a static configuration that enumerates the local services. That makes sense when the services are statically assigned to hosts. Once services are assigned dynamically, you must already have some kind of service discovery regime. In that case, I would expect that most people would switch to an Envoy sidecar (whose listener is the configured endpoint for the service in your service discovery).

To unsubscribe from this group and stop receiving emails from it, send an email to envoy-dev+unsubscribe@googlegroups.com.

To post to this group, send email to envo...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/envoy-dev/c20075f4-c940-4acf-a320-f0fdb145719d%40googlegroups.com.

[Pragya Gupta]

how can we split the EDS response into 3 and make it read those file individually by envoy 

To view this discussion on the web visit https://groups.google.com/d/msgid/envoy-dev/c20075f4-c940-4acf-a320-f0fdb145719d%40googlegroups.com.