Consul connect and external services

[Roman Rusakov]

Hey!

Congratulations to all who participated in this awesome new feature, we are still trying to understand the real impact of such option to our infrustructure scheme and development process :)

Fast question about external services (https://www.consul.io/docs/guides/external.html) - what is the plan to deal with them in the consul connect world? 

For example - will I be able to access some extenal service registered on another edge node from some other node in a standard way? 

Thanks,

Roman

[pba...@hashicorp.com]

Hey Roman,

I touched on this at a high level in my talk at HashiDays and it's a situation we've considered. The relevant section is here https://youtu.be/KZIu33sbwQQ?t=1149 and has a graphic if that helps but there are more details given below.

It's actually not quite possible with the current release to make this work with the built-in proxy but only for lack of a few simple config options that were not in scope for the primary localhost only use case. In practice we hope to release Envoy support soon which will be able to do this anyway and is probably a preferable solution in every respect. I'll attempt to cover roughly how below, but it's a use-case we will need better tooling and specific docs for going forward.

The proposal from the talk is roughly this:

 1. Provision one or more (for redundancy/throughput) nodes that are dedicated "proxies for external service X"

 2. On each one run a consul client agent (necessary since the proxy itself needs a local agent to get efficient auth calls and config from)

 3. Register an external service (optionally using consul-esm to run health checks) representing the external resource

 4. Register an unmanaged proxy on the proxy host agent with target service as the external service name

 5. Configure the proxy (assuming it's Envoy in future when supported) to proxy not to localhost but to the external service endpoint. You likely need to configure TLS and authentication tokens here somehow too.

At this point you can grant access to that external resource with an intention like `ALLOW web => hosted-db` and then web services will be allowed to connect to that proxy and through to the external service.

You obviously need to ensure nothing else can connect to the external service directly to make that meaningful and how you do that will depend on the service and platform - you might only provide credentials for the external service to those dedicated proxy nodes, or something more robust with a separate VPC or subnet and controlled routing/ip whitelisting etc.

This is how it works for a singe external service. If you have many external services you either replicate this with dedicated proxies for each (most secure as it limits blast radius of access of one proxy host is compromised) or you can colocate proxies for different services to reduce cost of dedicated instance per external service.

Medium term we want to make this more first class but will probably wait til Envoy integration is done rather than build more stuff on the built-in proxy. I can imagine at least that means documenting how to set it up thoroughly but might involve tooling or options that make it simpler too.

Hope this helps.

Paul

[Roman Rusakov]

Thank you Paul!

It seems reasonable. 

Looking forward to see it in action soon :)

btw I was thinking about repalcing our internal nginx to envoyproxy and now you are giving me such a chance :)

the only blocker for us may be Windows support  - https://github.com/envoyproxy/envoy/issues/129 - as now we will need to install proxy as a sidecar on every application host (windows unfortunately :( ) 

Thanks,

Roman

четверг, 19 июля 2018 г., 14:03:18 UTC+3 пользователь pba...@hashicorp.com написал:

[Valere Jeantet]

Hello, is there any tutorial, guide or an example of such a situation where you want an external service to be available through a proxy ?

Thanks a lot.

[Blake Covarrubias]

HI Valere,

We are working on an upcoming feature for Consul 1.8 (est. GA end of May) called Terminating Gateways which are designed to enable connectivity to external services (see hashicorp/consul#6357 for a bit more info). We will be publishing docs and tutorials when this is released.

Does this seem like it might satisfy your use case?

—

Blake Covarrubias

On Apr 12, 2020, at 12:47 PM, Valere Jeantet <valere....@gmail.com> wrote:

Hello, is there any tutorial, guide or an example of such a situation where you want an external service to be available through a proxy ?

Thanks a lot.

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.

GitHub Issues: https://github.com/hashicorp/consul/issues
Community chat: https://gitter.im/hashicorp-consul/Lobby
---
You received this message because you are subscribed to the Google Groups "Consul" group.
To unsubscribe from this group and stop receiving emails from it, send an email to consul-tool...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/consul-tool/99646c78-233e-4be8-b77d-0424400d90fb%40googlegroups.com.