Service Discovery and Docker containers

[Dudley Carr]

I'm trying to see if it's possible to include envoy in each Docker container and also have the various containers in an Envoy mesh using service discovery. The service discovery mechanism relies only on hostnames which seem to preclude the envoy per docker instance setup. Is there a way to support this use case at the moment?

[Matt Klein]

With the currently supported service discovery methods, this would require using the discovery service API:

https://lyft.github.io/envoy/docs/configuration/cluster_manager/sds_api.html

But beyond that there is nothing preventing Envoy from running inside each container (and just discovering IP/port pairs).

What service discovery system do you use now? FWIW, there is this PR open now against our discovery service which would make it a lot more pluggable then it is now.

Matt

On Sat, Oct 8, 2016 at 4:10 PM, Dudley Carr <dudle...@gmail.com> wrote:

I'm trying to see if it's possible to include envoy in each Docker container and also have the various containers in an Envoy mesh using service discovery. The service discovery mechanism relies only on hostnames which seem to preclude the envoy per docker instance setup. Is there a way to support this use case at the moment?

--
You received this message because you are subscribed to the Google Groups "Envoy Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to envoy-users+unsubscribe@googlegroups.com.
To post to this group, send email to envoy...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/envoy-users/380205ba-c8c0-4f9a-a84d-ae21716bf7ed%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Matt Klein

Software Engineer

mkl...@lyft.com / 206.327.4515

[Dudley Carr]

Matt,

Thanks for the reply, Matt! I misunderstood what is supposed to get exposed by the service discovery. I do have another point of confusion: I'm trying to configure a service to service configuration. It's unclear to me how two envoy discover one another. Potentially another source of confusion is what host / port pair a process should register with the Service Discovery Service if it intends other processes to reach it via the locally running envoy process.

Thanks again,

Dudley

[Matt Klein]

Dudley,

Envoy currently supports 4 different service discovery methods:

https://lyft.github.io/envoy/docs/intro/arch_overview/service_discovery.html#supported-service-discovery-types

How it is configured is dependent on your individual deployment. Can you potentially provide more info on what topology you are trying to create and we can advise on the best way of setting it up?

Thanks,

Matt

[Dudley Carr]

I’m trying to use the Sevice Discovery Service type. Our in-house registry is exposing the API as needed by Envoy. Here’s the’s the configuration that I have:

Host A (192.168.8.210)
 |
 +--> Docker Container (-p 8080:8080 -p 9211:9211)
      |
      +--> Envoy on port 9211, 9001 + Service 1 on port 8080

Host B (192.168.8.211)
 |
 +--> Docker Container (-p 8080:8080 -p 9211:9211)
      |
      +--> Envoy on 9211, 9001 + Service 2 on port 8080

Host C
 |
 +--> SDS (port 1111)

SDS returns the following:

curl http://c:1111/v1/registration/service1
{"ip_address": "192.168.8.210", "port": 8080}

curl http://c:1111/v1/registration/service2
{"ip_address": "192.168.8.211", "port": 8080}

Note: I’ve tried having the SDS return 9211 instead of 8080, but Envoy complains about the upstream services being unhealthy.

Service 1 is dependent on Service 2. Service 1 is attempting to reach Service 2 via the local envoy on port 9001 with the Host header set.

Thanks again.

​

[Matt Klein]

Hi Dudley,

The response is not quite right:

It should look like:

{

  "hosts": [

    {"ip_address": "192.168.8.210", "port": 8080}

  ]

}

I think you want to expose 9211 as the port from SDS. If you are still having issues after the above change can you pop into the Gitter room and we can debug in real time.

Thanks,

Matt

[Dudley Carr]

Matt,

Doh! That did the trick and thanks for the suggestion to use port 9211 as the port to be advertised via service discovery. 

Thanks again,

Dudley