Consul in multiple AWS VPCs

[Brian Lalor]

I’m trying to get a grip on how to configure multiple Consul data centers within AWS using VPCs.  I’m using a DC-per-VPC model and currently have half a dozen clusters/VPCs up and running independently.  Now I want to join them together.

In order to have the servers be able to talk to other servers, they’ll have to advertise their public IP addresses, which is a configuration option.  No problem there.  And I’ll need to set up the per-VPC “consul-wan” security groups so that each server in each DC can reach every other server in every other DC, using ports 8300 (TCP) and 8302 (TCP and UDP).  A fair number of entries, but so far so good.

The problem I’m running into is that — from the perspective of the server nodes — all agents are contacting the servers via the agents’ public IP addresses, for which there are no security group entries.

Would someone with a working multi-VPC/DC cluster setup mind sharing their security group settings?  Are you just opening up ports 8300 and 8302 to everyone and enabling encryption?  There’s just no way I will — or even can — put the public IP for every agent into the SG for the masters.

Thanks,

Brian

--

Brian Lalor

bla...@bravo5.org

[Brian Lalor]

To clarify, I also need to connect clusters in other regions, cloud providers, and data centers. Other solutions seem to assume vpc peering or vpns between these networks, which isn't necessary or desired. Looks like I'm probably going to have to remove all restrictions to the servers and rely on encryption to keep things safe. I'm not comfortable with that just yet. Hoping https://github.com/hashicorp/consul/issues/527 is implemented soon!

--
Brian Lalor
bla...@bravo5.org

--
You received this message because you are subscribed to the Google Groups "Consul" group.
To unsubscribe from this group and stop receiving emails from it, send an email to consul-tool...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Pedro Melo]

Hi,

I assume that you are using VPC peering aggrements?

That's what we use…

Bye,

--
You received this message because you are subscribed to the Google Groups "Consul" group.
To unsubscribe from this group and stop receiving emails from it, send an email to consul-tool...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Pedro Melo
@pedromelo
http://www.simplicidade.org/
xmpp:me...@simplicidade.org
mailto:me...@simplicidade.org

[Tiru Srikantha]

I'm running in an AWS VPC and my settings are like this:

I'm advertising the public IPs of each Consul master.
I have port 8301 TCP/UDP open to 0.0.0.0/0 on all servers.
I have port 8300-8302 TCP/UDP open on just the consul masters, again to world.

An alternative way to do this that I'm going to move to shortly now that 0.5.0 is out:
Port 8301 TCP/UDP open to 10.0.0.0/16 (Assuming that's your VPC subnet) on all servers.
Port 8301 TCP/UDP open on all servers to the array of public ips of your consul servers (give them EIPs and use some method like https://github.com/skymill/aws-ec2-assign-elastic-ip to lock them to a set of EIPs)
Port 8300 TCP open on Consul masters to 10.0.0.0/16 + [array of all consul servers in all your environments] (RPC calls over 8300 are forwarded with the master nodes in the current datacenter acting as a proxy - see: http://www.consul.io/docs/guides/datacenters.html )
Port 8301 TCP/UDP open to 10.0.0.0/16 on all Consul masters.
Port 8302 TCP/UDP open to [array of all consul servers in all your environments]. Again, EIPs are your friend here.

Encryption should be on pretty much always in my opinion. All Consul masters advertise their public IP, all non-Masters advertise their internal IP.

Two things that make this work:
1. RPC calls are proxied between Consul servers in different datacenters, so each Consul master only needs to be able to talk to a set comprised of [all Consul masters in all datacenters] + [all servers in the local datacenter].
2. As of v0.5.0 you can define a service-specific IP for each service so you don't need to advertise the node's public IP any longer just to expose 1 service: http://www.consul.io/docs/agent/services.html